Buckets:

Mercity
/

Desicion-Model-Custom

Files
xet
Mercity/Desicion-Model-Custom / think-decide /logs /notes_full.log
Pranav2748
8 days ago
download
raw
409 kB
{"selected": 10000, "by_source": {"clinc": 1250, "snli": 2250, "arc": 250, "sgd": 1000, "synthetic_constraints": 500, "synthetic_ordinal": 1125, "synthetic_probability": 375, "boolq": 1000, "dbpedia": 1500, "goemotions": 750}}
W1106 14:52:12.485000 29398 torch/_opaque_base.py:6] torch._opaque_base is deprecated, use torch._custom_class_base instead
W1106 14:52:12.487000 29398 torch/_library/opaque_object.py:288] register_opaque_type is deprecated, use register_custom_class instead
W1106 14:52:12.488000 29398 torch/_library/opaque_object.py:211] typ='value' is deprecated, use typ='constant' instead
(MainProcess pid=29398) INFO 11-06 14:52:19 [startup.py:96] non-default args: {'dtype': 'bfloat16', 'max_model_len': 8192, 'enable_prefix_caching': True, 'gpu_memory_utilization': 0.6, 'max_num_seqs': 256, 'disable_log_stats': True, 'model': '/root/hf/qwen3.5-9b'}
(MainProcess pid=29398) INFO 11-06 14:52:19 [model.py:744] Resolved architecture: Qwen3_5ForConditionalGeneration
(MainProcess pid=29398) INFO 11-06 14:52:19 [model.py:2170] Using max model len 8192
(MainProcess pid=29398) INFO 11-06 14:52:22 [scheduler.py:310] Chunked prefill is enabled with max_num_batched_tokens=8192.
(MainProcess pid=29398) INFO 11-06 14:52:22 [config.py:766] Mamba cache mode is set to 'align' for Qwen3_5ForConditionalGeneration by default when prefix caching is enabled
(MainProcess pid=29398) INFO 11-06 14:52:22 [kernel.py:480] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])
(MainProcess pid=29398) WARNING 11-06 14:52:33 [system_utils.py:154] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
W1106 14:52:37.625000 29757 torch/_opaque_base.py:6] torch._opaque_base is deprecated, use torch._custom_class_base instead
W1106 14:52:37.626000 29757 torch/_library/opaque_object.py:288] register_opaque_type is deprecated, use register_custom_class instead
W1106 14:52:37.627000 29757 torch/_library/opaque_object.py:211] typ='value' is deprecated, use typ='constant' instead
(EngineCore pid=29757) INFO 11-06 14:52:43 [core.py:139] Initializing a V1 LLM engine (v0.33.0) with config: model='/root/hf/qwen3.5-9b', speculative_config=None, tokenizer='/root/hf/qwen3.5-9b', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=8192, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, quantization_config=None, enforce_eager=False, aux_output_config=AuxOutputConfig(enable_return_routed_experts=False, max_bytes=None), kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, per_request_spec_decode_metrics='none', kv_cache_metrics=False, kv_cache_metrics_sample=0.01, custom_histogram_buckets=None, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=0, served_model_name=/root/hf/qwen3.5-9b, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::qwen4_exp_ple_short_conv', 'vllm::qwen4_exp_qsa_with_output', 'vllm::qwen4_exp_qsa_prepare_with_output', 'vllm::linear_attention', 'vllm::qwen_gdn_attention_core', 'vllm::qwen_gdn_attention_core_fused_norm_packed', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::hpc_rope_norm_forward', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'cudagraph_decoder_replay': True, 'decoder_replay_cudagraph_capture_sizes': [], 'decoder_replay_trim_threshold': 768, 'compile_sizes': [], 'compile_ranges_endpoints': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'fuse_allreduce_rms': False, 'enable_qk_norm_rope_fusion': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False, 'fuse_qk_norm_rope_kvcache': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native']), enable_flashinfer_autotune=True, enable_rocm_segmented_attn_autotune=False, enable_cutedsl_warmup=True, enable_jit_warmup=True, monokernel=False, moe_backend='auto', sparse_indexer_topk_backend='auto', linear_backend='auto', linear_backend_per_quant=None)
(EngineCore pid=29757) INFO 11-06 14:52:45 [parallel_state.py:1929] world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_3b3adca44b3f4f4db079778215f44b9a backend=nccl
(EngineCore pid=29757) INFO 11-06 14:52:45 [parallel_state.py:2385] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, ETP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=29757) INFO 11-06 14:52:45 [gpu_worker.py:513] Using V2 Model Runner
(EngineCore pid=29757) INFO 11-06 14:52:46 [model_runner.py:399] Loading model from scratch...
(EngineCore pid=29757) INFO 11-06 14:52:47 [cuda.py:567] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(EngineCore pid=29757) INFO 11-06 14:52:47 [mm_encoder_attention.py:372] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
(EngineCore pid=29757) INFO 11-06 14:52:47 [qwen_gdn_linear_attn.py:300] Using Triton/FLA GDN prefill kernel (requested=auto, head_k_dim=128).
(EngineCore pid=29757) INFO 11-06 14:52:48 [cuda.py:508] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION'].
(EngineCore pid=29757) INFO 11-06 14:52:48 [flash_attn.py:1122] Using FlashAttention version 2
(EngineCore pid=29757) INFO 11-06 14:52:48 [base_loader.py:82] Initializing model took 2.010216 seconds
(EngineCore pid=29757) INFO 11-06 14:52:48 [weight_utils.py:1144] Filesystem type for checkpoints: OVERLAY. Checkpoint size: 17.98 GiB. Available RAM: 68.22 GiB.
(EngineCore pid=29757) INFO 11-06 14:52:48 [weight_utils.py:1167] Auto-prefetch is disabled because the filesystem (OVERLAY) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.

Loading safetensors checkpoint shards:   0% Completed | 0/4 [00:00<?, ?it/s]

Loading safetensors checkpoint shards:  25% Completed | 1/4 [00:00<00:01,  1.62it/s]

Loading safetensors checkpoint shards:  50% Completed | 2/4 [00:01<00:01,  1.48it/s]

Loading safetensors checkpoint shards:  75% Completed | 3/4 [00:02<00:00,  1.34it/s]
(EngineCore pid=29757) WARNING 11-06 14:52:51 [utils.py:333] Attempted to load weight layers.0.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:51 [utils.py:333] Attempted to load weight layers.1.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:51 [utils.py:333] Attempted to load weight layers.10.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:51 [utils.py:333] Attempted to load weight layers.12.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:51 [utils.py:333] Attempted to load weight layers.13.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:51 [utils.py:333] Attempted to load weight layers.14.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:51 [utils.py:333] Attempted to load weight layers.16.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:51 [utils.py:333] Attempted to load weight layers.17.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:51 [utils.py:333] Attempted to load weight layers.18.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:51 [utils.py:333] Attempted to load weight layers.2.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:51 [utils.py:333] Attempted to load weight layers.20.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
[transformers] Qwen3VL video processing does not apply the per-frame pixel cap the reference implementation (qwen-vl-utils) applies, so some videos cost far more tokens than they would there. In v5.22 the capped behavior will become the default and `cap_pixels_per_frame` will be removed. Pass `cap_pixels_per_frame=True` to adopt the reference behavior now, or `False` to keep the current behavior and silence this warning.
(EngineCore pid=29757) WARNING 11-06 14:52:52 [utils.py:333] Attempted to load weight layers.21.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:52 [utils.py:333] Attempted to load weight layers.22.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:52 [utils.py:333] Attempted to load weight layers.24.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:52 [utils.py:333] Attempted to load weight layers.25.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:52 [utils.py:333] Attempted to load weight layers.26.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:52 [utils.py:333] Attempted to load weight layers.28.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:52 [utils.py:333] Attempted to load weight layers.29.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:52 [utils.py:333] Attempted to load weight layers.30.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:52 [utils.py:333] Attempted to load weight layers.4.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:52 [utils.py:333] Attempted to load weight layers.5.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(MainProcess pid=29398) INFO 11-06 14:52:52 [base.py:283] Multi-modal warmup completed in 19.306s
(EngineCore pid=29757) WARNING 11-06 14:52:52 [utils.py:333] Attempted to load weight layers.6.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:52 [utils.py:333] Attempted to load weight layers.8.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=29757) WARNING 11-06 14:52:52 [utils.py:333] Attempted to load weight layers.9.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16

Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:05<00:00,  1.61s/it]

Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:05<00:00,  1.28s/it]

(EngineCore pid=29757) INFO 11-06 14:52:54 [default_loader.py:506] Loading weights took 5.24 seconds
(EngineCore pid=29757) INFO 11-06 14:52:55 [model_runner.py:428] Model loading took 17.57 GiB memory and 9.065548 seconds
(EngineCore pid=29757) INFO 11-06 14:52:55 [topk_topp_sampler.py:149] Using FlashInfer for top-p & top-k sampling.
(EngineCore pid=29757) INFO 11-06 14:52:55 [torch_utils.py:281] Reducing Torch threads from 255 to 1 for serving. Set OMP_NUM_THREADS in the external environment to override.
(EngineCore pid=29757) INFO 11-06 14:52:55 [interface.py:1108] Setting attention block size to 528 tokens to ensure that attention page size is >= mamba page size.
(EngineCore pid=29757) INFO 11-06 14:52:55 [interface.py:1132] Padding mamba page size by 0.76% to ensure that mamba page size and attention page size are exactly equal.
(EngineCore pid=29757) INFO 11-06 14:52:55 [utils.py:381] Using LBNHC KV cache layout.
(EngineCore pid=29757) INFO 11-06 14:53:03 [encoder_runner.py:131] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
(EngineCore pid=29757) INFO 11-06 14:53:13 [caching.py:342] reconstructed serializable fn from standalone compile artifacts. num_artifacts=13 num_submods=33
(EngineCore pid=29757) INFO 11-06 14:53:13 [decorators.py:326] [backbone] Directly load AOT compilation from path /root/.cache/vllm/torch_compile_cache/torch_aot_compile/570fe35d686514fcab15b2f5a7f595cad3825ba85c7952aa6d2232b00327468d/rank_0_0/model
(EngineCore pid=29757) INFO 11-06 14:53:13 [monitor.py:62] [backbone] torch.compile took 0.69 s in total
(EngineCore pid=29757) INFO 11-06 14:53:15 [monitor.py:91] [backbone] Initial profiling/warmup run took 1.54 s

Capturing CUDA graphs (PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):   2%|▏         | 1/51 [00:01<00:50,  1.01s/it]
Capturing CUDA graphs (PIECEWISE):   4%|▍         | 2/51 [00:01<00:43,  1.14it/s]
Capturing CUDA graphs (PIECEWISE):   6%|▌         | 3/51 [00:02<00:38,  1.26it/s]
Capturing CUDA graphs (PIECEWISE):   8%|▊         | 4/51 [00:03<00:39,  1.19it/s]
Capturing CUDA graphs (PIECEWISE):  10%|▉         | 5/51 [00:04<00:34,  1.32it/s]
Capturing CUDA graphs (PIECEWISE):  12%|█▏        | 6/51 [00:04<00:31,  1.44it/s]
Capturing CUDA graphs (PIECEWISE):  14%|█▎        | 7/51 [00:05<00:30,  1.46it/s]
Capturing CUDA graphs (PIECEWISE):  16%|█▌        | 8/51 [00:05<00:27,  1.54it/s]
Capturing CUDA graphs (PIECEWISE):  18%|█▊        | 9/51 [00:06<00:26,  1.58it/s]
Capturing CUDA graphs (PIECEWISE):  20%|█▉        | 10/51 [00:06<00:24,  1.69it/s]
Capturing CUDA graphs (PIECEWISE):  22%|██▏       | 11/51 [00:07<00:22,  1.80it/s]
Capturing CUDA graphs (PIECEWISE):  24%|██▎       | 12/51 [00:07<00:20,  1.87it/s]
Capturing CUDA graphs (PIECEWISE):  25%|██▌       | 13/51 [00:08<00:20,  1.88it/s]
Capturing CUDA graphs (PIECEWISE):  27%|██▋       | 14/51 [00:08<00:19,  1.93it/s]
Capturing CUDA graphs (PIECEWISE):  29%|██▉       | 15/51 [00:09<00:18,  1.94it/s]
Capturing CUDA graphs (PIECEWISE):  31%|███▏      | 16/51 [00:09<00:16,  2.18it/s]
Capturing CUDA graphs (PIECEWISE):  33%|███▎      | 17/51 [00:09<00:12,  2.73it/s]
Capturing CUDA graphs (PIECEWISE):  35%|███▌      | 18/51 [00:10<00:10,  3.10it/s]
Capturing CUDA graphs (PIECEWISE):  37%|███▋      | 19/51 [00:10<00:08,  3.63it/s]
Capturing CUDA graphs (PIECEWISE):  39%|███▉      | 20/51 [00:10<00:07,  4.21it/s]
Capturing CUDA graphs (PIECEWISE):  41%|████      | 21/51 [00:10<00:06,  4.75it/s]
Capturing CUDA graphs (PIECEWISE):  43%|████▎     | 22/51 [00:10<00:05,  5.10it/s]
Capturing CUDA graphs (PIECEWISE):  45%|████▌     | 23/51 [00:10<00:05,  5.57it/s]
Capturing CUDA graphs (PIECEWISE):  47%|████▋     | 24/51 [00:11<00:04,  5.91it/s]
Capturing CUDA graphs (PIECEWISE):  49%|████▉     | 25/51 [00:11<00:04,  5.82it/s]
Capturing CUDA graphs (PIECEWISE):  51%|█████     | 26/51 [00:11<00:04,  6.08it/s]
Capturing CUDA graphs (PIECEWISE):  53%|█████▎    | 27/51 [00:11<00:03,  6.26it/s]
Capturing CUDA graphs (PIECEWISE):  55%|█████▍    | 28/51 [00:11<00:03,  6.35it/s]
Capturing CUDA graphs (PIECEWISE):  57%|█████▋    | 29/51 [00:11<00:03,  6.53it/s]
Capturing CUDA graphs (PIECEWISE):  59%|█████▉    | 30/51 [00:11<00:03,  6.68it/s]
Capturing CUDA graphs (PIECEWISE):  61%|██████    | 31/51 [00:12<00:03,  6.36it/s]
Capturing CUDA graphs (PIECEWISE):  63%|██████▎   | 32/51 [00:12<00:03,  6.25it/s]
Capturing CUDA graphs (PIECEWISE):  65%|██████▍   | 33/51 [00:12<00:02,  6.51it/s]
Capturing CUDA graphs (PIECEWISE):  67%|██████▋   | 34/51 [00:12<00:02,  6.33it/s]
Capturing CUDA graphs (PIECEWISE):  69%|██████▊   | 35/51 [00:12<00:02,  6.48it/s]
Capturing CUDA graphs (PIECEWISE):  71%|███████   | 36/51 [00:12<00:02,  6.74it/s]
Capturing CUDA graphs (PIECEWISE):  73%|███████▎  | 37/51 [00:12<00:02,  6.85it/s]
Capturing CUDA graphs (PIECEWISE):  75%|███████▍  | 38/51 [00:13<00:01,  6.76it/s]
Capturing CUDA graphs (PIECEWISE):  76%|███████▋  | 39/51 [00:13<00:01,  7.10it/s]
Capturing CUDA graphs (PIECEWISE):  78%|███████▊  | 40/51 [00:13<00:01,  7.27it/s]
Capturing CUDA graphs (PIECEWISE):  80%|████████  | 41/51 [00:13<00:01,  6.18it/s]
Capturing CUDA graphs (PIECEWISE):  82%|████████▏ | 42/51 [00:13<00:01,  6.59it/s]
Capturing CUDA graphs (PIECEWISE):  84%|████████▍ | 43/51 [00:13<00:01,  6.99it/s]
Capturing CUDA graphs (PIECEWISE):  86%|████████▋ | 44/51 [00:13<00:00,  7.27it/s]
Capturing CUDA graphs (PIECEWISE):  88%|████████▊ | 45/51 [00:14<00:00,  6.93it/s]
Capturing CUDA graphs (PIECEWISE):  90%|█████████ | 46/51 [00:14<00:00,  7.15it/s]
Capturing CUDA graphs (PIECEWISE):  92%|█████████▏| 47/51 [00:14<00:00,  7.51it/s]
Capturing CUDA graphs (PIECEWISE):  94%|█████████▍| 48/51 [00:14<00:00,  7.48it/s]
Capturing CUDA graphs (PIECEWISE):  96%|█████████▌| 49/51 [00:14<00:00,  7.45it/s]
Capturing CUDA graphs (PIECEWISE):  98%|█████████▊| 50/51 [00:14<00:00,  7.64it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:14<00:00,  7.73it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:14<00:00,  3.42it/s]

Capturing CUDA graphs (FULL):   0%|          | 0/2 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 2/2 [00:00<00:00, 13.76it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 2/2 [00:00<00:00, 13.73it/s]
(EngineCore pid=29757) INFO 11-06 14:53:33 [model_runner.py:1163] Graph capturing finished in 16 secs, took 0.43 GiB
(EngineCore pid=29757) INFO 11-06 14:53:34 [gpu_worker.py:772] Available KV cache memory: 27.35 GiB
(EngineCore pid=29757) INFO 11-06 14:53:34 [gpu_worker.py:787] CUDA graph memory profiling is enabled (default since v0.21.0). The current --gpu-memory-utilization=0.6000 is equivalent to --gpu-memory-utilization=0.5940 without CUDA graph memory profiling. To maintain the same effective KV cache size as before, increase --gpu-memory-utilization to 0.6060. To disable, set VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0.
(EngineCore pid=29757) INFO 11-06 14:53:34 [kv_cache_utils.py:1147] GPU KV cache size: 631,901 tokens, Maximum concurrency for 8,192 tokens per request: 77.14x
(EngineCore pid=29757) INFO 11-06 14:53:34 [kv_cache_utils.py:865] kv cache group sizes (528, 528, 528, 528)
(EngineCore pid=29757) INFO 11-06 14:53:34 [kv_cache_utils.py:866] kv lcm block sizes 528
(EngineCore pid=29757) INFO 11-06 14:53:35 [kernel_warmup.py:179] JIT kernel warmup starting.
(EngineCore pid=29757) INFO 11-06 14:53:35 [kernel_warmup.py:192] JIT kernel warmup finished in 0.00s.
(EngineCore pid=29757) INFO 11-06 14:53:35 [qwen_triton_warmup.py:370] Warming up Qwen GDN Triton kernels for model_type=qwen3_5_text.
(EngineCore pid=29757) INFO 11-06 14:53:35 [qwen_vl_triton_warmup.py:57] Warmed position embedding and vision rotary kernels on grids=[(1, 16, 16), (1, 16, 2), (1, 2, 16), (1, 2, 2)].
(EngineCore pid=29757) INFO 11-06 14:53:35 [mamba_triton_warmup.py:42] Warmed Mamba batch_memcpy_kernel.

Capturing CUDA graphs (PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):   2%|▏         | 1/51 [00:00<00:40,  1.23it/s]
Capturing CUDA graphs (PIECEWISE):   4%|▍         | 2/51 [00:01<00:38,  1.27it/s]
Capturing CUDA graphs (PIECEWISE):   6%|▌         | 3/51 [00:02<00:35,  1.35it/s]
Capturing CUDA graphs (PIECEWISE):   8%|▊         | 4/51 [00:02<00:32,  1.43it/s]
Capturing CUDA graphs (PIECEWISE):  10%|▉         | 5/51 [00:03<00:32,  1.44it/s]
Capturing CUDA graphs (PIECEWISE):  12%|█▏        | 6/51 [00:04<00:29,  1.51it/s]
Capturing CUDA graphs (PIECEWISE):  14%|█▎        | 7/51 [00:04<00:28,  1.55it/s]
Capturing CUDA graphs (PIECEWISE):  16%|█▌        | 8/51 [00:05<00:27,  1.58it/s]
Capturing CUDA graphs (PIECEWISE):  18%|█▊        | 9/51 [00:06<00:26,  1.59it/s]
Capturing CUDA graphs (PIECEWISE):  20%|█▉        | 10/51 [00:06<00:24,  1.64it/s]
Capturing CUDA graphs (PIECEWISE):  22%|██▏       | 11/51 [00:07<00:23,  1.73it/s]
Capturing CUDA graphs (PIECEWISE):  24%|██▎       | 12/51 [00:07<00:22,  1.77it/s]
Capturing CUDA graphs (PIECEWISE):  25%|██▌       | 13/51 [00:08<00:21,  1.75it/s]
Capturing CUDA graphs (PIECEWISE):  27%|██▋       | 14/51 [00:08<00:20,  1.84it/s]
Capturing CUDA graphs (PIECEWISE):  29%|██▉       | 15/51 [00:09<00:17,  2.03it/s]
Capturing CUDA graphs (PIECEWISE):  31%|███▏      | 16/51 [00:09<00:14,  2.34it/s]
Capturing CUDA graphs (PIECEWISE):  33%|███▎      | 17/51 [00:09<00:11,  2.93it/s]
Capturing CUDA graphs (PIECEWISE):  35%|███▌      | 18/51 [00:09<00:09,  3.47it/s]
Capturing CUDA graphs (PIECEWISE):  37%|███▋      | 19/51 [00:09<00:07,  4.06it/s]
Capturing CUDA graphs (PIECEWISE):  39%|███▉      | 20/51 [00:09<00:06,  4.62it/s]
Capturing CUDA graphs (PIECEWISE):  41%|████      | 21/51 [00:10<00:05,  5.02it/s]
Capturing CUDA graphs (PIECEWISE):  43%|████▎     | 22/51 [00:10<00:05,  5.53it/s]
Capturing CUDA graphs (PIECEWISE):  45%|████▌     | 23/51 [00:10<00:04,  5.93it/s]
Capturing CUDA graphs (PIECEWISE):  47%|████▋     | 24/51 [00:10<00:04,  5.96it/s]
Capturing CUDA graphs (PIECEWISE):  49%|████▉     | 25/51 [00:10<00:04,  6.19it/s]
Capturing CUDA graphs (PIECEWISE):  51%|█████     | 26/51 [00:10<00:03,  6.47it/s]
Capturing CUDA graphs (PIECEWISE):  53%|█████▎    | 27/51 [00:10<00:03,  7.00it/s]
Capturing CUDA graphs (PIECEWISE):  55%|█████▍    | 28/51 [00:11<00:03,  6.23it/s]
Capturing CUDA graphs (PIECEWISE):  57%|█████▋    | 29/51 [00:11<00:03,  6.41it/s]
Capturing CUDA graphs (PIECEWISE):  59%|█████▉    | 30/51 [00:11<00:03,  6.60it/s]
Capturing CUDA graphs (PIECEWISE):  61%|██████    | 31/51 [00:11<00:03,  6.56it/s]
Capturing CUDA graphs (PIECEWISE):  63%|██████▎   | 32/51 [00:11<00:02,  6.67it/s]
Capturing CUDA graphs (PIECEWISE):  65%|██████▍   | 33/51 [00:11<00:02,  6.91it/s]
Capturing CUDA graphs (PIECEWISE):  67%|██████▋   | 34/51 [00:12<00:02,  6.88it/s]
Capturing CUDA graphs (PIECEWISE):  69%|██████▊   | 35/51 [00:12<00:02,  6.98it/s]
Capturing CUDA graphs (PIECEWISE):  71%|███████   | 36/51 [00:12<00:02,  7.23it/s]
Capturing CUDA graphs (PIECEWISE):  73%|███████▎  | 37/51 [00:12<00:01,  7.43it/s]
Capturing CUDA graphs (PIECEWISE):  75%|███████▍  | 38/51 [00:12<00:01,  6.96it/s]
Capturing CUDA graphs (PIECEWISE):  76%|███████▋  | 39/51 [00:12<00:01,  7.24it/s]
Capturing CUDA graphs (PIECEWISE):  78%|███████▊  | 40/51 [00:12<00:01,  7.48it/s]
Capturing CUDA graphs (PIECEWISE):  80%|████████  | 41/51 [00:12<00:01,  7.29it/s]
Capturing CUDA graphs (PIECEWISE):  82%|████████▏ | 42/51 [00:13<00:01,  7.34it/s]
Capturing CUDA graphs (PIECEWISE):  84%|████████▍ | 43/51 [00:13<00:01,  7.40it/s]
Capturing CUDA graphs (PIECEWISE):  86%|████████▋ | 44/51 [00:13<00:00,  7.51it/s]
Capturing CUDA graphs (PIECEWISE):  88%|████████▊ | 45/51 [00:13<00:00,  7.30it/s]
Capturing CUDA graphs (PIECEWISE):  90%|█████████ | 46/51 [00:13<00:00,  7.39it/s]
Capturing CUDA graphs (PIECEWISE):  92%|█████████▏| 47/51 [00:13<00:00,  7.71it/s]
Capturing CUDA graphs (PIECEWISE):  94%|█████████▍| 48/51 [00:13<00:00,  7.86it/s]
Capturing CUDA graphs (PIECEWISE):  96%|█████████▌| 49/51 [00:14<00:00,  7.50it/s]
Capturing CUDA graphs (PIECEWISE):  98%|█████████▊| 50/51 [00:14<00:00,  7.58it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:14<00:00,  7.76it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:14<00:00,  3.57it/s]

Capturing CUDA graphs (FULL):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL):   6%|▌         | 2/35 [00:00<00:02, 13.51it/s]
Capturing CUDA graphs (FULL):  11%|█▏        | 4/35 [00:00<00:02, 12.76it/s]
Capturing CUDA graphs (FULL):  17%|█▋        | 6/35 [00:00<00:02, 13.18it/s]
Capturing CUDA graphs (FULL):  23%|██▎       | 8/35 [00:00<00:01, 13.71it/s]
Capturing CUDA graphs (FULL):  29%|██▊       | 10/35 [00:00<00:01, 13.26it/s]
Capturing CUDA graphs (FULL):  34%|███▍      | 12/35 [00:00<00:01, 13.79it/s]
Capturing CUDA graphs (FULL):  40%|████      | 14/35 [00:01<00:01, 14.01it/s]
Capturing CUDA graphs (FULL):  46%|████▌     | 16/35 [00:01<00:01, 13.89it/s]
Capturing CUDA graphs (FULL):  51%|█████▏    | 18/35 [00:01<00:01, 14.37it/s]
Capturing CUDA graphs (FULL):  57%|█████▋    | 20/35 [00:01<00:00, 15.10it/s]
Capturing CUDA graphs (FULL):  63%|██████▎   | 22/35 [00:01<00:00, 15.71it/s]
Capturing CUDA graphs (FULL):  69%|██████▊   | 24/35 [00:01<00:00, 15.92it/s]
Capturing CUDA graphs (FULL):  74%|███████▍  | 26/35 [00:01<00:00, 16.15it/s]
Capturing CUDA graphs (FULL):  80%|████████  | 28/35 [00:01<00:00, 16.28it/s]
Capturing CUDA graphs (FULL):  86%|████████▌ | 30/35 [00:02<00:00, 17.02it/s]
Capturing CUDA graphs (FULL):  91%|█████████▏| 32/35 [00:02<00:00, 17.02it/s]
Capturing CUDA graphs (FULL):  97%|█████████▋| 34/35 [00:02<00:00, 17.32it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 35/35 [00:02<00:00, 15.36it/s]
(EngineCore pid=29757) INFO 11-06 14:53:53 [model_runner.py:1163] Graph capturing finished in 18 secs, took 0.27 GiB
(EngineCore pid=29757) INFO 11-06 14:53:53 [gpu_worker.py:1003] CUDA graph pool memory: 0.27 GiB (actual), 0.48 GiB (estimated), difference: 0.21 GiB (78.8%).
(EngineCore pid=29757) INFO 11-06 14:53:53 [gpu_worker.py:1066] Free memory on device (74.84/79.25 GiB) on startup. Desired GPU memory utilization is (0.6, 47.55 GiB). Actual usage is 17.94 GiB for consumed memory (weights + non-torch), 2.27 GiB for peak activation, and 0.27 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=28917212263` (26.93 GiB) to fit into requested memory, or `--kv-cache-memory=58217283072` (54.22 GiB) to fully utilize gpu memory. Current kv cache memory in use is 27.35 GiB.
(EngineCore pid=29757) INFO 11-06 14:53:54 [jit_monitor.py:84] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn.
(EngineCore pid=29757) INFO 11-06 14:53:55 [core.py:405] init engine (profile, create kv cache, warmup model) took 59.41 s (compilation: 0.69 s)
(EngineCore pid=29757) INFO 11-06 14:53:55 [kernel.py:480] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])
(MainProcess pid=29398) INFO 11-06 14:53:55 [hf.py:736] Detected the chat template content format to be 'openai'. You can set `--chat-template-content-format` to override this.

Rendering prompts:   0%|          | 0/10000 [00:00<?, ?it/s]
Rendering prompts:   1%|          | 118/10000 [00:00<00:08, 1172.42it/s]
Rendering prompts:   2%|▏         | 239/10000 [00:00<00:08, 1194.43it/s]
Rendering prompts:   4%|▎         | 371/10000 [00:00<00:07, 1248.72it/s]
Rendering prompts:   5%|▍         | 496/10000 [00:00<00:07, 1212.92it/s]
Rendering prompts:   6%|▋         | 643/10000 [00:00<00:07, 1300.84it/s]
Rendering prompts:   8%|▊         | 793/10000 [00:00<00:06, 1367.51it/s]
Rendering prompts:   9%|▉         | 945/10000 [00:00<00:06, 1414.50it/s]
Rendering prompts:  11%|█         | 1098/10000 [00:00<00:06, 1447.95it/s]
Rendering prompts:  12%|█▎        | 1250/10000 [00:00<00:05, 1469.13it/s]
Rendering prompts:  14%|█▍        | 1403/10000 [00:01<00:05, 1487.64it/s]
Rendering prompts:  16%|█▌        | 1552/10000 [00:01<00:06, 1293.83it/s]
Rendering prompts:  17%|█▋        | 1686/10000 [00:01<00:08, 996.73it/s] 
Rendering prompts:  18%|█▊        | 1799/10000 [00:01<00:08, 925.15it/s]
Rendering prompts:  19%|█▉        | 1913/10000 [00:01<00:08, 973.60it/s]
Rendering prompts:  20%|██        | 2023/10000 [00:01<00:07, 1002.38it/s]
Rendering prompts:  21%|██▏       | 2130/10000 [00:01<00:09, 834.67it/s] 
Rendering prompts:  22%|██▏       | 2223/10000 [00:02<00:09, 856.25it/s]
Rendering prompts:  23%|██▎       | 2325/10000 [00:02<00:08, 895.07it/s]
Rendering prompts:  24%|██▍       | 2446/10000 [00:02<00:07, 976.44it/s]
Rendering prompts:  26%|██▌       | 2556/10000 [00:02<00:07, 1010.07it/s]
Rendering prompts:  27%|██▋       | 2661/10000 [00:02<00:17, 426.36it/s] 
Rendering prompts:  27%|██▋       | 2740/10000 [00:03<00:28, 255.18it/s]
Rendering prompts:  28%|██▊       | 2799/10000 [00:04<00:34, 207.37it/s]
Rendering prompts:  28%|██▊       | 2844/10000 [00:04<00:42, 168.43it/s]
Rendering prompts:  29%|██▉       | 2878/10000 [00:04<00:45, 157.10it/s]
Rendering prompts:  29%|██▉       | 2906/10000 [00:05<00:42, 168.06it/s]
Rendering prompts:  29%|██▉       | 2933/10000 [00:05<00:43, 161.45it/s]
Rendering prompts:  30%|██▉       | 2956/10000 [00:05<00:41, 169.44it/s]
Rendering prompts:  30%|██▉       | 2979/10000 [00:05<00:45, 154.39it/s]
Rendering prompts:  30%|██▉       | 2999/10000 [00:05<00:50, 138.39it/s]
Rendering prompts:  30%|███       | 3016/10000 [00:05<00:55, 125.67it/s]
Rendering prompts:  30%|███       | 3031/10000 [00:06<00:54, 128.51it/s]
Rendering prompts:  30%|███       | 3046/10000 [00:06<01:03, 109.30it/s]
Rendering prompts:  31%|███       | 3059/10000 [00:06<01:11, 97.67it/s] 
Rendering prompts:  31%|███       | 3075/10000 [00:06<01:03, 109.07it/s]
Rendering prompts:  31%|███       | 3091/10000 [00:06<00:57, 119.78it/s]
Rendering prompts:  31%|███       | 3105/10000 [00:06<00:55, 124.32it/s]
Rendering prompts:  31%|███       | 3122/10000 [00:06<00:50, 135.38it/s]
Rendering prompts:  31%|███▏      | 3137/10000 [00:06<00:49, 138.86it/s]
Rendering prompts:  32%|███▏      | 3152/10000 [00:07<00:49, 138.07it/s]
Rendering prompts:  32%|███▏      | 3167/10000 [00:07<01:00, 112.90it/s]
Rendering prompts:  32%|███▏      | 3180/10000 [00:07<00:59, 115.04it/s]
Rendering prompts:  32%|███▏      | 3193/10000 [00:07<00:57, 117.45it/s]
Rendering prompts:  32%|███▏      | 3206/10000 [00:07<01:09, 97.08it/s] 
Rendering prompts:  32%|███▏      | 3217/10000 [00:07<01:08, 98.96it/s]
Rendering prompts:  32%|███▏      | 3230/10000 [00:07<01:03, 106.51it/s]
Rendering prompts:  32%|███▏      | 3243/10000 [00:08<01:13, 92.02it/s] 
Rendering prompts:  33%|███▎      | 3256/10000 [00:08<01:07, 100.06it/s]
Rendering prompts:  33%|███▎      | 3268/10000 [00:08<01:04, 103.97it/s]
Rendering prompts:  33%|███▎      | 3280/10000 [00:08<01:17, 87.16it/s] 
Rendering prompts:  33%|███▎      | 3296/10000 [00:08<01:04, 103.86it/s]
Rendering prompts:  33%|███▎      | 3308/10000 [00:08<01:02, 106.76it/s]
Rendering prompts:  33%|███▎      | 3320/10000 [00:08<01:16, 87.10it/s] 
Rendering prompts:  33%|███▎      | 3330/10000 [00:09<01:30, 73.53it/s]
Rendering prompts:  33%|███▎      | 3345/10000 [00:09<01:14, 89.38it/s]
Rendering prompts:  34%|███▎      | 3356/10000 [00:09<01:26, 77.09it/s]
Rendering prompts:  34%|███▎      | 3367/10000 [00:09<01:19, 83.82it/s]
Rendering prompts:  34%|███▍      | 3377/10000 [00:09<01:17, 85.55it/s]
Rendering prompts:  34%|███▍      | 3392/10000 [00:09<01:18, 83.83it/s]
Rendering prompts:  34%|███▍      | 3401/10000 [00:09<01:21, 81.44it/s]
Rendering prompts:  34%|███▍      | 3410/10000 [00:10<01:34, 69.44it/s]
Rendering prompts:  34%|███▍      | 3422/10000 [00:10<01:22, 79.77it/s]
Rendering prompts:  34%|███▍      | 3431/10000 [00:10<01:36, 67.85it/s]
Rendering prompts:  34%|███▍      | 3447/10000 [00:10<01:14, 87.73it/s]
Rendering prompts:  35%|███▍      | 3457/10000 [00:10<01:12, 89.94it/s]
Rendering prompts:  35%|███▍      | 3467/10000 [00:10<01:11, 91.60it/s]
Rendering prompts:  35%|███▍      | 3477/10000 [00:10<01:25, 75.90it/s]
Rendering prompts:  35%|███▍      | 3488/10000 [00:10<01:17, 83.61it/s]
Rendering prompts:  35%|███▍      | 3498/10000 [00:11<01:15, 86.43it/s]
Rendering prompts:  35%|███▌      | 3515/10000 [00:11<01:00, 107.81it/s]
Rendering prompts:  35%|███▌      | 3527/10000 [00:11<01:15, 85.36it/s] 
Rendering prompts:  35%|███▌      | 3537/10000 [00:11<01:26, 75.09it/s]
Rendering prompts:  36%|███▌      | 3551/10000 [00:11<01:12, 89.08it/s]
Rendering prompts:  36%|███▌      | 3573/10000 [00:11<00:53, 119.99it/s]
Rendering prompts:  36%|███▌      | 3593/10000 [00:11<00:45, 139.80it/s]
Rendering prompts:  36%|███▌      | 3609/10000 [00:12<00:56, 112.33it/s]
Rendering prompts:  36%|███▌      | 3623/10000 [00:12<01:17, 82.56it/s] 
Rendering prompts:  36%|███▋      | 3637/10000 [00:12<01:08, 93.07it/s]
Rendering prompts:  36%|███▋      | 3649/10000 [00:12<01:04, 97.99it/s]
Rendering prompts:  37%|███▋      | 3661/10000 [00:12<01:15, 83.74it/s]
Rendering prompts:  37%|███▋      | 3671/10000 [00:12<01:13, 86.65it/s]
Rendering prompts:  37%|███▋      | 3681/10000 [00:12<01:10, 89.61it/s]
Rendering prompts:  37%|███▋      | 3691/10000 [00:13<01:09, 91.36it/s]
Rendering prompts:  37%|███▋      | 3706/10000 [00:13<00:59, 106.52it/s]
Rendering prompts:  37%|███▋      | 3718/10000 [00:13<01:11, 87.81it/s] 
Rendering prompts:  37%|███▋      | 3731/10000 [00:13<01:04, 97.53it/s]
Rendering prompts:  38%|███▊      | 3750/10000 [00:13<00:51, 120.31it/s]
Rendering prompts:  38%|███▊      | 3764/10000 [00:13<00:50, 124.30it/s]
Rendering prompts:  38%|███▊      | 3778/10000 [00:13<00:48, 128.07it/s]
Rendering prompts:  38%|███▊      | 3792/10000 [00:13<00:47, 130.51it/s]
Rendering prompts:  38%|███▊      | 3806/10000 [00:13<00:46, 132.31it/s]
Rendering prompts:  38%|███▊      | 3832/10000 [00:14<00:44, 138.15it/s]
Rendering prompts:  38%|███▊      | 3847/10000 [00:14<00:43, 140.23it/s]
Rendering prompts:  39%|███▊      | 3862/10000 [00:14<00:44, 139.30it/s]
Rendering prompts:  39%|███▉      | 3876/10000 [00:14<00:54, 112.75it/s]
Rendering prompts:  39%|███▉      | 3890/10000 [00:14<00:51, 118.95it/s]
Rendering prompts:  39%|███▉      | 3905/10000 [00:14<00:48, 126.46it/s]
Rendering prompts:  39%|███▉      | 3926/10000 [00:14<00:41, 147.82it/s]
Rendering prompts:  39%|███▉      | 3942/10000 [00:15<00:52, 116.12it/s]
Rendering prompts:  40%|███▉      | 3964/10000 [00:15<00:43, 139.95it/s]
Rendering prompts:  40%|███▉      | 3980/10000 [00:15<00:50, 118.92it/s]
Rendering prompts:  40%|███▉      | 3994/10000 [00:15<00:49, 122.39it/s]
Rendering prompts:  40%|████      | 4013/10000 [00:15<00:43, 138.53it/s]
Rendering prompts:  40%|████      | 4029/10000 [00:15<00:42, 139.99it/s]
Rendering prompts:  40%|████      | 4044/10000 [00:15<00:50, 117.64it/s]
Rendering prompts:  41%|████      | 4057/10000 [00:15<00:49, 119.34it/s]
Rendering prompts:  41%|████      | 4070/10000 [00:16<00:50, 117.84it/s]
Rendering prompts:  41%|████      | 4088/10000 [00:16<00:44, 133.77it/s]
Rendering prompts:  41%|████      | 4103/10000 [00:16<00:51, 113.65it/s]
Rendering prompts:  41%|████▏     | 4125/10000 [00:16<00:42, 138.73it/s]
Rendering prompts:  41%|████▏     | 4141/10000 [00:16<00:42, 139.30it/s]
Rendering prompts:  42%|████▏     | 4156/10000 [00:16<00:42, 139.00it/s]
Rendering prompts:  42%|████▏     | 4171/10000 [00:16<00:50, 115.82it/s]
Rendering prompts:  42%|████▏     | 4193/10000 [00:16<00:41, 140.50it/s]
Rendering prompts:  42%|████▏     | 4209/10000 [00:17<00:40, 144.04it/s]
Rendering prompts:  42%|████▏     | 4232/10000 [00:17<00:34, 166.46it/s]
Rendering prompts:  42%|████▎     | 4250/10000 [00:17<00:34, 168.75it/s]
Rendering prompts:  43%|████▎     | 4268/10000 [00:17<00:51, 111.68it/s]
Rendering prompts:  43%|████▎     | 4283/10000 [00:17<00:57, 99.86it/s] 
Rendering prompts:  43%|████▎     | 4297/10000 [00:17<00:53, 107.57it/s]
Rendering prompts:  43%|████▎     | 4310/10000 [00:18<01:01, 92.50it/s] 
Rendering prompts:  43%|████▎     | 4321/10000 [00:18<00:59, 95.73it/s]
Rendering prompts:  43%|████▎     | 4332/10000 [00:18<00:57, 98.54it/s]
Rendering prompts:  43%|████▎     | 4344/10000 [00:18<00:54, 103.43it/s]
Rendering prompts:  44%|████▎     | 4356/10000 [00:18<01:04, 86.92it/s] 
Rendering prompts:  44%|████▎     | 4366/10000 [00:18<01:03, 88.92it/s]
Rendering prompts:  44%|████▍     | 4376/10000 [00:18<01:02, 90.41it/s]
Rendering prompts:  44%|████▍     | 4386/10000 [00:18<01:01, 91.68it/s]
Rendering prompts:  44%|████▍     | 4401/10000 [00:19<01:04, 87.31it/s]
Rendering prompts:  44%|████▍     | 4411/10000 [00:19<01:02, 89.25it/s]
Rendering prompts:  44%|████▍     | 4425/10000 [00:19<00:54, 101.93it/s]
Rendering prompts:  44%|████▍     | 4436/10000 [00:19<00:55, 100.91it/s]
Rendering prompts:  45%|████▍     | 4452/10000 [00:19<00:47, 115.60it/s]
Rendering prompts:  45%|████▍     | 4464/10000 [00:19<00:57, 96.25it/s] 
Rendering prompts:  45%|████▍     | 4475/10000 [00:19<00:56, 98.54it/s]
Rendering prompts:  45%|████▍     | 4486/10000 [00:19<00:54, 101.30it/s]
Rendering prompts:  45%|████▌     | 4504/10000 [00:19<00:44, 122.30it/s]
Rendering prompts:  45%|████▌     | 4517/10000 [00:20<01:07, 80.63it/s] 
Rendering prompts:  45%|████▌     | 4528/10000 [00:20<01:03, 85.69it/s]
Rendering prompts:  46%|████▌     | 4553/10000 [00:20<00:44, 122.40it/s]
Rendering prompts:  46%|████▌     | 4568/10000 [00:20<00:42, 129.03it/s]
Rendering prompts:  46%|████▌     | 4588/10000 [00:20<00:36, 147.07it/s]
Rendering prompts:  46%|████▌     | 4605/10000 [00:20<00:35, 151.89it/s]
Rendering prompts:  46%|████▌     | 4622/10000 [00:20<00:43, 123.02it/s]
Rendering prompts:  46%|████▋     | 4639/10000 [00:21<00:39, 134.04it/s]
Rendering prompts:  47%|████▋     | 4654/10000 [00:21<00:47, 112.90it/s]
Rendering prompts:  47%|████▋     | 4667/10000 [00:21<00:45, 116.54it/s]
Rendering prompts:  47%|████▋     | 4685/10000 [00:21<00:40, 132.02it/s]
Rendering prompts:  47%|████▋     | 4703/10000 [00:21<00:36, 144.23it/s]
Rendering prompts:  47%|████▋     | 4719/10000 [00:21<00:35, 147.50it/s]
Rendering prompts:  47%|████▋     | 4735/10000 [00:21<00:35, 150.33it/s]
Rendering prompts:  48%|████▊     | 4751/10000 [00:21<00:35, 146.89it/s]
Rendering prompts:  48%|████▊     | 4767/10000 [00:22<00:42, 122.88it/s]
Rendering prompts:  48%|████▊     | 4786/10000 [00:22<00:37, 139.03it/s]
Rendering prompts:  48%|████▊     | 4801/10000 [00:22<00:37, 138.19it/s]
Rendering prompts:  48%|████▊     | 4817/10000 [00:22<00:36, 143.81it/s]
Rendering prompts:  48%|████▊     | 4833/10000 [00:22<00:34, 148.17it/s]
Rendering prompts:  48%|████▊     | 4849/10000 [00:22<00:34, 151.28it/s]
Rendering prompts:  49%|████▊     | 4865/10000 [00:22<00:33, 153.44it/s]
Rendering prompts:  49%|████▉     | 4881/10000 [00:22<00:41, 122.40it/s]
Rendering prompts:  49%|████▉     | 4895/10000 [00:22<00:40, 125.27it/s]
Rendering prompts:  49%|████▉     | 4909/10000 [00:23<00:39, 128.91it/s]
Rendering prompts:  50%|█████     | 5024/10000 [00:23<00:12, 402.89it/s]
Rendering prompts:  51%|█████▏    | 5135/10000 [00:23<00:08, 593.19it/s]
Rendering prompts:  53%|█████▎    | 5255/10000 [00:23<00:06, 764.12it/s]
Rendering prompts:  54%|█████▎    | 5373/10000 [00:23<00:05, 881.28it/s]
Rendering prompts:  55%|█████▍    | 5481/10000 [00:23<00:04, 937.05it/s]
Rendering prompts:  56%|█████▌    | 5597/10000 [00:23<00:04, 1001.15it/s]
Rendering prompts:  57%|█████▋    | 5712/10000 [00:23<00:04, 1041.00it/s]
Rendering prompts:  58%|█████▊    | 5831/10000 [00:23<00:03, 1084.14it/s]
Rendering prompts:  60%|█████▉    | 5965/10000 [00:23<00:03, 1159.72it/s]
Rendering prompts:  61%|██████    | 6097/10000 [00:24<00:03, 1205.09it/s]
Rendering prompts:  62%|██████▏   | 6229/10000 [00:24<00:03, 1237.43it/s]
Rendering prompts:  64%|██████▎   | 6364/10000 [00:24<00:02, 1270.86it/s]
Rendering prompts:  65%|██████▌   | 6501/10000 [00:24<00:02, 1298.10it/s]
Rendering prompts:  66%|██████▋   | 6632/10000 [00:24<00:03, 1052.79it/s]
Rendering prompts:  67%|██████▋   | 6745/10000 [00:24<00:03, 915.39it/s] 
Rendering prompts:  68%|██████▊   | 6845/10000 [00:24<00:04, 701.77it/s]
Rendering prompts:  69%|██████▉   | 6928/10000 [00:25<00:05, 613.95it/s]
Rendering prompts:  70%|██████▉   | 6999/10000 [00:25<00:04, 623.44it/s]
Rendering prompts:  71%|███████   | 7069/10000 [00:25<00:04, 635.98it/s]
Rendering prompts:  71%|███████▏  | 7138/10000 [00:25<00:04, 639.45it/s]
Rendering prompts:  72%|███████▏  | 7206/10000 [00:25<00:04, 646.03it/s]
Rendering prompts:  73%|███████▎  | 7274/10000 [00:25<00:04, 653.69it/s]
Rendering prompts:  74%|███████▍  | 7419/10000 [00:25<00:02, 869.15it/s]
Rendering prompts:  76%|███████▌  | 7563/10000 [00:25<00:02, 1027.63it/s]
Rendering prompts:  77%|███████▋  | 7700/10000 [00:25<00:02, 1124.60it/s]
Rendering prompts:  78%|███████▊  | 7839/10000 [00:26<00:01, 1198.53it/s]
Rendering prompts:  80%|███████▉  | 7981/10000 [00:26<00:01, 1261.91it/s]
Rendering prompts:  81%|████████  | 8121/10000 [00:26<00:01, 1300.55it/s]
Rendering prompts:  83%|████████▎ | 8264/10000 [00:26<00:01, 1337.94it/s]
Rendering prompts:  84%|████████▍ | 8399/10000 [00:26<00:01, 1339.42it/s]
Rendering prompts:  85%|████████▌ | 8541/10000 [00:26<00:01, 1363.21it/s]
Rendering prompts:  87%|████████▋ | 8690/10000 [00:26<00:00, 1398.59it/s]
Rendering prompts:  88%|████████▊ | 8831/10000 [00:26<00:00, 1345.21it/s]
Rendering prompts:  90%|████████▉ | 8967/10000 [00:26<00:00, 1236.85it/s]
Rendering prompts:  91%|█████████ | 9093/10000 [00:27<00:00, 1137.17it/s]
Rendering prompts:  92%|█████████▏| 9210/10000 [00:27<00:00, 1091.52it/s]
Rendering prompts:  93%|█████████▎| 9321/10000 [00:27<00:00, 1062.59it/s]
Rendering prompts:  94%|█████████▍| 9429/10000 [00:27<00:00, 1032.69it/s]
Rendering prompts:  95%|█████████▌| 9533/10000 [00:27<00:00, 1007.88it/s]
Rendering prompts:  96%|█████████▋| 9635/10000 [00:27<00:00, 1001.01it/s]
Rendering prompts:  97%|█████████▋| 9736/10000 [00:27<00:00, 993.59it/s] 
Rendering prompts:  99%|█████████▊| 9852/10000 [00:27<00:00, 1038.79it/s]
Rendering prompts: 100%|█████████▉| 9978/10000 [00:27<00:00, 1101.58it/s]
Rendering prompts: 100%|██████████| 10000/10000 [00:27<00:00, 358.67it/s]

Processed prompts:   0%|          | 0/10000 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts:   1%|          | 78/10000 [00:00<01:24, 117.88it/s, est. speed input: 35088.20 toks/s, output: 7422.05 toks/s]
Processed prompts:   1%|          | 90/10000 [00:01<02:42, 61.12it/s, est. speed input: 20392.50 toks/s, output: 4468.24 toks/s] 
Processed prompts:   1%|          | 96/10000 [00:01<03:19, 49.65it/s, est. speed input: 18057.89 toks/s, output: 3890.95 toks/s]
Processed prompts:   1%|          | 102/10000 [00:02<05:18, 31.04it/s, est. speed input: 13672.41 toks/s, output: 2958.92 toks/s]
Processed prompts:   1%|          | 112/10000 [00:02<06:07, 26.93it/s, est. speed input: 12089.15 toks/s, output: 2658.37 toks/s]
Processed prompts:   1%|          | 123/10000 [00:03<06:34, 25.03it/s, est. speed input: 11351.28 toks/s, output: 2483.79 toks/s]
Processed prompts:   1%|▏         | 137/10000 [00:04<07:51, 20.92it/s, est. speed input: 9810.96 toks/s, output: 2205.15 toks/s] 
Processed prompts:   2%|▏         | 154/10000 [00:05<08:25, 19.49it/s, est. speed input: 8827.78 toks/s, output: 2035.00 toks/s]
Processed prompts:   2%|▏         | 164/10000 [00:06<11:05, 14.79it/s, est. speed input: 7490.45 toks/s, output: 1768.82 toks/s]
Processed prompts:   2%|▏         | 172/10000 [00:07<12:12, 13.41it/s, est. speed input: 6960.14 toks/s, output: 1655.06 toks/s]
Processed prompts:   2%|▏         | 184/10000 [00:07<11:05, 14.75it/s, est. speed input: 6829.32 toks/s, output: 1638.27 toks/s]
Processed prompts:   2%|▏         | 193/10000 [00:08<12:40, 12.90it/s, est. speed input: 6349.14 toks/s, output: 1537.83 toks/s]
Processed prompts:   2%|▏         | 205/10000 [00:09<12:01, 13.58it/s, est. speed input: 6182.93 toks/s, output: 1511.04 toks/s]
Processed prompts:   2%|▏         | 212/10000 [00:10<14:33, 11.21it/s, est. speed input: 5830.81 toks/s, output: 1416.83 toks/s]
Processed prompts:   2%|▏         | 219/10000 [00:11<14:26, 11.29it/s, est. speed input: 5772.99 toks/s, output: 1390.12 toks/s]
Processed prompts:   2%|▏         | 222/10000 [00:11<17:01,  9.57it/s, est. speed input: 5505.99 toks/s, output: 1334.62 toks/s]
Processed prompts:   2%|▏         | 225/10000 [00:12<17:53,  9.10it/s, est. speed input: 5424.42 toks/s, output: 1309.31 toks/s]
Processed prompts:   2%|▏         | 232/10000 [00:12<14:01, 11.61it/s, est. speed input: 5432.81 toks/s, output: 1327.48 toks/s]
Processed prompts:   2%|▏         | 237/10000 [00:12<14:42, 11.06it/s, est. speed input: 5366.54 toks/s, output: 1307.91 toks/s]
Processed prompts:   2%|▏         | 242/10000 [00:13<14:46, 11.01it/s, est. speed input: 5341.88 toks/s, output: 1294.28 toks/s]
Processed prompts:   2%|▏         | 244/10000 [00:13<17:40,  9.20it/s, est. speed input: 5197.43 toks/s, output: 1264.63 toks/s]
Processed prompts:   2%|▏         | 246/10000 [00:14<18:24,  8.83it/s, est. speed input: 5148.16 toks/s, output: 1252.10 toks/s]
Processed prompts:   2%|▏         | 248/10000 [00:14<20:44,  7.84it/s, est. speed input: 5041.04 toks/s, output: 1231.28 toks/s]
Processed prompts:   2%|▏         | 249/10000 [00:14<24:10,  6.72it/s, est. speed input: 4960.54 toks/s, output: 1212.05 toks/s]
Processed prompts:   3%|▎         | 251/10000 [00:15<21:43,  7.48it/s, est. speed input: 4948.11 toks/s, output: 1207.00 toks/s]
Processed prompts:   3%|▎         | 252/10000 [00:15<26:13,  6.19it/s, est. speed input: 4858.25 toks/s, output: 1188.22 toks/s]
Processed prompts:   3%|▎         | 253/10000 [00:15<28:13,  5.76it/s, est. speed input: 4809.16 toks/s, output: 1177.01 toks/s]
Processed prompts:   3%|▎         | 256/10000 [00:16<34:12,  4.75it/s, est. speed input: 4667.55 toks/s, output: 1134.69 toks/s]
Processed prompts:   3%|▎         | 257/10000 [00:16<39:58,  4.06it/s, est. speed input: 4565.43 toks/s, output: 1112.19 toks/s]
Processed prompts:   3%|▎         | 258/10000 [00:16<36:42,  4.42it/s, est. speed input: 4538.08 toks/s, output: 1108.39 toks/s]
Processed prompts:   3%|▎         | 259/10000 [00:17<37:54,  4.28it/s, est. speed input: 4500.35 toks/s, output: 1093.79 toks/s]
Processed prompts:   3%|▎         | 260/10000 [00:18<1:14:37,  2.18it/s, est. speed input: 4219.37 toks/s, output: 1026.36 toks/s]
Processed prompts:   3%|▎         | 261/10000 [00:19<1:36:01,  1.69it/s, est. speed input: 4033.32 toks/s, output: 978.25 toks/s] 
Processed prompts:   3%|▎         | 262/10000 [00:19<1:25:59,  1.89it/s, est. speed input: 3969.71 toks/s, output: 963.82 toks/s]
Processed prompts:   3%|▎         | 264/10000 [00:20<1:05:16,  2.49it/s, est. speed input: 3894.12 toks/s, output: 947.96 toks/s]
Processed prompts:   3%|▎         | 266/10000 [00:20<51:36,  3.14it/s, est. speed input: 3841.01 toks/s, output: 936.98 toks/s]  
Processed prompts:   3%|▎         | 267/10000 [00:20<52:49,  3.07it/s, est. speed input: 3784.16 toks/s, output: 924.62 toks/s]
Processed prompts:   3%|▎         | 269/10000 [00:21<37:20,  4.34it/s, est. speed input: 3794.84 toks/s, output: 923.66 toks/s]
Processed prompts:   3%|▎         | 270/10000 [00:21<40:09,  4.04it/s, est. speed input: 3746.25 toks/s, output: 913.35 toks/s]
Processed prompts:   3%|▎         | 271/10000 [00:21<40:16,  4.03it/s, est. speed input: 3714.24 toks/s, output: 905.66 toks/s]
Processed prompts:   3%|▎         | 272/10000 [00:21<42:01,  3.86it/s, est. speed input: 3675.55 toks/s, output: 896.61 toks/s]
Processed prompts:   3%|▎         | 275/10000 [00:22<27:13,  5.95it/s, est. speed input: 3667.95 toks/s, output: 895.98 toks/s]
Processed prompts:   3%|▎         | 281/10000 [00:22<15:10, 10.68it/s, est. speed input: 3686.12 toks/s, output: 903.20 toks/s]
Processed prompts:   3%|▎         | 285/10000 [00:22<14:37, 11.07it/s, est. speed input: 3664.24 toks/s, output: 900.47 toks/s]
Processed prompts:   3%|▎         | 289/10000 [00:23<14:52, 10.89it/s, est. speed input: 3652.46 toks/s, output: 897.16 toks/s]
Processed prompts:   3%|▎         | 291/10000 [00:23<17:32,  9.22it/s, est. speed input: 3629.12 toks/s, output: 889.69 toks/s]
Processed prompts:   3%|▎         | 296/10000 [00:23<13:16, 12.19it/s, est. speed input: 3652.27 toks/s, output: 894.20 toks/s]
Processed prompts:   3%|▎         | 298/10000 [00:24<18:12,  8.88it/s, est. speed input: 3601.99 toks/s, output: 881.88 toks/s]
Processed prompts:   3%|▎         | 302/10000 [00:24<16:27,  9.82it/s, est. speed input: 3592.85 toks/s, output: 881.09 toks/s]
Processed prompts:   3%|▎         | 308/10000 [00:24<12:35, 12.83it/s, est. speed input: 3637.94 toks/s, output: 886.86 toks/s]
Processed prompts:   3%|▎         | 313/10000 [00:25<12:02, 13.41it/s, est. speed input: 3649.42 toks/s, output: 888.75 toks/s]
Processed prompts:   3%|▎         | 320/10000 [00:25<10:03, 16.05it/s, est. speed input: 3696.06 toks/s, output: 896.81 toks/s]
Processed prompts:   3%|▎         | 326/10000 [00:25<10:05, 15.98it/s, est. speed input: 3724.91 toks/s, output: 899.92 toks/s]
Processed prompts:   3%|▎         | 332/10000 [00:26<09:41, 16.63it/s, est. speed input: 3735.14 toks/s, output: 904.33 toks/s]
Processed prompts:   3%|▎         | 342/10000 [00:26<07:45, 20.75it/s, est. speed input: 3825.52 toks/s, output: 920.10 toks/s]
Processed prompts:   3%|▎         | 348/10000 [00:26<09:03, 17.76it/s, est. speed input: 3824.72 toks/s, output: 920.72 toks/s]
Processed prompts:   4%|▎         | 363/10000 [00:27<06:18, 25.48it/s, est. speed input: 3913.56 toks/s, output: 949.96 toks/s]
Processed prompts:   4%|▍         | 375/10000 [00:27<06:54, 23.21it/s, est. speed input: 3944.45 toks/s, output: 960.41 toks/s]
Processed prompts:   4%|▍         | 378/10000 [00:28<07:01, 22.85it/s, est. speed input: 3941.36 toks/s, output: 963.21 toks/s]
Processed prompts:   4%|▍         | 384/10000 [00:28<07:53, 20.33it/s, est. speed input: 3952.30 toks/s, output: 964.87 toks/s]
Processed prompts:   4%|▍         | 392/10000 [00:28<08:08, 19.68it/s, est. speed input: 3962.18 toks/s, output: 969.84 toks/s]
Processed prompts:   4%|▍         | 407/10000 [00:29<06:22, 25.11it/s, est. speed input: 4054.93 toks/s, output: 991.92 toks/s]
Processed prompts:   4%|▍         | 413/10000 [00:29<08:20, 19.14it/s, est. speed input: 4023.96 toks/s, output: 985.68 toks/s]
Processed prompts:   4%|▍         | 422/10000 [00:30<07:33, 21.11it/s, est. speed input: 4068.95 toks/s, output: 997.58 toks/s]
Processed prompts:   4%|▍         | 425/10000 [00:30<09:31, 16.74it/s, est. speed input: 4039.73 toks/s, output: 991.13 toks/s]
Processed prompts:   4%|▍         | 436/10000 [00:30<07:07, 22.37it/s, est. speed input: 4076.82 toks/s, output: 1010.56 toks/s]
Processed prompts:   4%|▍         | 441/10000 [00:31<08:44, 18.22it/s, est. speed input: 4061.65 toks/s, output: 1007.71 toks/s]
Processed prompts:   5%|▍         | 451/10000 [00:31<07:17, 21.81it/s, est. speed input: 4115.36 toks/s, output: 1021.81 toks/s]
Processed prompts:   5%|▍         | 459/10000 [00:32<07:46, 20.44it/s, est. speed input: 4135.29 toks/s, output: 1026.16 toks/s]
Processed prompts:   5%|▍         | 464/10000 [00:32<08:49, 18.00it/s, est. speed input: 4124.77 toks/s, output: 1025.33 toks/s]
Processed prompts:   5%|▍         | 473/10000 [00:32<07:36, 20.86it/s, est. speed input: 4144.19 toks/s, output: 1037.12 toks/s]
Processed prompts:   5%|▍         | 476/10000 [00:33<09:50, 16.13it/s, est. speed input: 4123.49 toks/s, output: 1030.34 toks/s]
Processed prompts:   5%|▍         | 478/10000 [00:33<10:17, 15.42it/s, est. speed input: 4115.63 toks/s, output: 1029.24 toks/s]
Processed prompts:   5%|▍         | 480/10000 [00:33<11:41, 13.57it/s, est. speed input: 4097.59 toks/s, output: 1026.17 toks/s]
Processed prompts:   5%|▍         | 482/10000 [00:34<15:22, 10.32it/s, est. speed input: 4063.68 toks/s, output: 1018.15 toks/s]
Processed prompts:   5%|▍         | 488/10000 [00:34<10:40, 14.85it/s, est. speed input: 4086.09 toks/s, output: 1026.82 toks/s]
Processed prompts:   5%|▍         | 490/10000 [00:34<13:11, 12.01it/s, est. speed input: 4060.16 toks/s, output: 1021.90 toks/s]
Processed prompts:   5%|▍         | 492/10000 [00:34<13:16, 11.93it/s, est. speed input: 4061.37 toks/s, output: 1021.63 toks/s]
Processed prompts:   5%|▍         | 495/10000 [00:34<13:07, 12.06it/s, est. speed input: 4050.36 toks/s, output: 1021.54 toks/s]
Processed prompts:   5%|▍         | 498/10000 [00:35<13:06, 12.09it/s, est. speed input: 4040.21 toks/s, output: 1020.85 toks/s]
Processed prompts:   5%|▌         | 500/10000 [00:35<19:45,  8.01it/s, est. speed input: 3988.66 toks/s, output: 1009.49 toks/s]
Processed prompts:   5%|▌         | 506/10000 [00:36<14:52, 10.63it/s, est. speed input: 3982.09 toks/s, output: 1013.07 toks/s]
Processed prompts:   5%|▌         | 508/10000 [00:36<21:36,  7.32it/s, est. speed input: 3923.60 toks/s, output: 999.69 toks/s] 
Processed prompts:   5%|▌         | 510/10000 [00:37<24:31,  6.45it/s, est. speed input: 3887.62 toks/s, output: 992.34 toks/s]
Processed prompts:   5%|▌         | 512/10000 [00:37<23:40,  6.68it/s, est. speed input: 3879.81 toks/s, output: 989.97 toks/s]
Processed prompts:   5%|▌         | 514/10000 [00:37<21:11,  7.46it/s, est. speed input: 3881.42 toks/s, output: 990.48 toks/s]
Processed prompts:   5%|▌         | 515/10000 [00:37<23:05,  6.85it/s, est. speed input: 3865.24 toks/s, output: 987.22 toks/s]
Processed prompts:   5%|▌         | 517/10000 [00:38<20:12,  7.82it/s, est. speed input: 3858.95 toks/s, output: 986.66 toks/s]
Processed prompts:   5%|▌         | 519/10000 [00:38<19:05,  8.27it/s, est. speed input: 3848.85 toks/s, output: 984.06 toks/s]
Processed prompts:   5%|▌         | 520/10000 [00:38<21:28,  7.36it/s, est. speed input: 3842.08 toks/s, output: 981.47 toks/s]
Processed prompts:   5%|▌         | 521/10000 [00:38<23:42,  6.66it/s, est. speed input: 3826.81 toks/s, output: 978.02 toks/s]
Processed prompts:   5%|▌         | 523/10000 [00:39<27:52,  5.67it/s, est. speed input: 3794.58 toks/s, output: 970.44 toks/s]
Processed prompts:   5%|▌         | 526/10000 [00:39<21:42,  7.27it/s, est. speed input: 3785.25 toks/s, output: 968.54 toks/s]
Processed prompts:   5%|▌         | 527/10000 [00:39<23:37,  6.68it/s, est. speed input: 3770.95 toks/s, output: 965.00 toks/s]
Processed prompts:   5%|▌         | 529/10000 [00:39<20:14,  7.80it/s, est. speed input: 3765.92 toks/s, output: 963.62 toks/s]
Processed prompts:   5%|▌         | 530/10000 [00:39<22:32,  7.00it/s, est. speed input: 3751.99 toks/s, output: 960.36 toks/s]
Processed prompts:   5%|▌         | 531/10000 [00:40<23:38,  6.68it/s, est. speed input: 3740.87 toks/s, output: 957.88 toks/s]
Processed prompts:   5%|▌         | 533/10000 [00:40<19:46,  7.98it/s, est. speed input: 3735.24 toks/s, output: 957.08 toks/s]
Processed prompts:   5%|▌         | 541/10000 [00:40<09:01, 17.48it/s, est. speed input: 3757.65 toks/s, output: 964.51 toks/s]
Processed prompts:   5%|▌         | 543/10000 [00:40<13:39, 11.54it/s, est. speed input: 3730.39 toks/s, output: 957.80 toks/s]
Processed prompts:   5%|▌         | 547/10000 [00:41<11:37, 13.56it/s, est. speed input: 3732.23 toks/s, output: 959.08 toks/s]
Processed prompts:   6%|▌         | 551/10000 [00:41<11:06, 14.17it/s, est. speed input: 3729.25 toks/s, output: 959.27 toks/s]
Processed prompts:   6%|▌         | 553/10000 [00:41<13:02, 12.08it/s, est. speed input: 3715.80 toks/s, output: 956.47 toks/s]
Processed prompts:   6%|▌         | 560/10000 [00:41<08:53, 17.68it/s, est. speed input: 3732.65 toks/s, output: 962.74 toks/s]
Processed prompts:   6%|▌         | 564/10000 [00:42<10:38, 14.79it/s, est. speed input: 3719.17 toks/s, output: 960.73 toks/s]
Processed prompts:   6%|▌         | 572/10000 [00:42<08:09, 19.26it/s, est. speed input: 3735.41 toks/s, output: 966.58 toks/s]
Processed prompts:   6%|▌         | 576/10000 [00:42<10:02, 15.65it/s, est. speed input: 3719.15 toks/s, output: 963.11 toks/s]
Processed prompts:   6%|▌         | 581/10000 [00:43<09:31, 16.48it/s, est. speed input: 3720.33 toks/s, output: 964.19 toks/s]
Processed prompts:   6%|▌         | 590/10000 [00:43<07:41, 20.41it/s, est. speed input: 3738.64 toks/s, output: 971.09 toks/s]
Processed prompts:   6%|▌         | 593/10000 [00:43<10:16, 15.27it/s, est. speed input: 3715.95 toks/s, output: 965.82 toks/s]
Processed prompts:   6%|▌         | 595/10000 [00:44<11:34, 13.54it/s, est. speed input: 3704.33 toks/s, output: 963.31 toks/s]
Processed prompts:   6%|▌         | 597/10000 [00:44<11:54, 13.16it/s, est. speed input: 3699.22 toks/s, output: 962.77 toks/s]
Processed prompts:   6%|▌         | 602/10000 [00:44<10:24, 15.06it/s, est. speed input: 3702.08 toks/s, output: 965.13 toks/s]
Processed prompts:   6%|▌         | 609/10000 [00:44<08:51, 17.67it/s, est. speed input: 3709.24 toks/s, output: 969.06 toks/s]
Processed prompts:   6%|▌         | 617/10000 [00:45<08:12, 19.05it/s, est. speed input: 3716.20 toks/s, output: 973.60 toks/s]
Processed prompts:   6%|▌         | 623/10000 [00:45<08:50, 17.69it/s, est. speed input: 3711.92 toks/s, output: 974.83 toks/s]
Processed prompts:   6%|▋         | 631/10000 [00:45<07:55, 19.72it/s, est. speed input: 3722.21 toks/s, output: 980.32 toks/s]
Processed prompts:   6%|▋         | 637/10000 [00:46<08:39, 18.02it/s, est. speed input: 3716.59 toks/s, output: 980.50 toks/s]
Processed prompts:   6%|▋         | 647/10000 [00:46<07:11, 21.68it/s, est. speed input: 3735.79 toks/s, output: 988.83 toks/s]
Processed prompts:   7%|▋         | 652/10000 [00:47<08:46, 17.75it/s, est. speed input: 3721.18 toks/s, output: 986.69 toks/s]
Processed prompts:   7%|▋         | 662/10000 [00:47<07:14, 21.51it/s, est. speed input: 3741.48 toks/s, output: 995.09 toks/s]
Processed prompts:   7%|▋         | 671/10000 [00:47<07:26, 20.89it/s, est. speed input: 3746.22 toks/s, output: 999.35 toks/s]
Processed prompts:   7%|▋         | 675/10000 [00:48<09:04, 17.13it/s, est. speed input: 3730.21 toks/s, output: 997.03 toks/s]
Processed prompts:   7%|▋         | 683/10000 [00:48<07:46, 19.96it/s, est. speed input: 3744.73 toks/s, output: 1004.05 toks/s]
Processed prompts:   7%|▋         | 695/10000 [00:49<06:42, 23.12it/s, est. speed input: 3764.67 toks/s, output: 1012.78 toks/s]
Processed prompts:   7%|▋         | 699/10000 [00:49<09:04, 17.08it/s, est. speed input: 3741.08 toks/s, output: 1008.02 toks/s]
Processed prompts:   7%|▋         | 701/10000 [00:49<10:17, 15.07it/s, est. speed input: 3729.57 toks/s, output: 1005.50 toks/s]
Processed prompts:   7%|▋         | 703/10000 [00:50<12:52, 12.03it/s, est. speed input: 3709.38 toks/s, output: 1000.91 toks/s]
Processed prompts:   7%|▋         | 706/10000 [00:50<11:59, 12.92it/s, est. speed input: 3709.75 toks/s, output: 1001.87 toks/s]
Processed prompts:   7%|▋         | 711/10000 [00:50<10:31, 14.72it/s, est. speed input: 3711.91 toks/s, output: 1004.54 toks/s]
Processed prompts:   7%|▋         | 714/10000 [00:50<11:40, 13.26it/s, est. speed input: 3702.35 toks/s, output: 1003.16 toks/s]
Processed prompts:   7%|▋         | 720/10000 [00:51<09:33, 16.19it/s, est. speed input: 3708.95 toks/s, output: 1007.26 toks/s]
Processed prompts:   7%|▋         | 724/10000 [00:51<10:27, 14.79it/s, est. speed input: 3700.98 toks/s, output: 1007.06 toks/s]
Processed prompts:   7%|▋         | 728/10000 [00:51<10:24, 14.84it/s, est. speed input: 3697.77 toks/s, output: 1008.04 toks/s]
Processed prompts:   7%|▋         | 732/10000 [00:52<10:23, 14.87it/s, est. speed input: 3694.76 toks/s, output: 1008.18 toks/s]
Processed prompts:   7%|▋         | 734/10000 [00:52<12:09, 12.70it/s, est. speed input: 3684.32 toks/s, output: 1006.04 toks/s]
Processed prompts:   7%|▋         | 736/10000 [00:52<12:50, 12.02it/s, est. speed input: 3678.85 toks/s, output: 1005.12 toks/s]
Processed prompts:   7%|▋         | 742/10000 [00:52<09:25, 16.37it/s, est. speed input: 3688.84 toks/s, output: 1010.33 toks/s]
Processed prompts:   7%|▋         | 744/10000 [00:53<12:14, 12.60it/s, est. speed input: 3674.06 toks/s, output: 1006.83 toks/s]
Processed prompts:   8%|▊         | 750/10000 [00:53<09:15, 16.66it/s, est. speed input: 3683.76 toks/s, output: 1011.74 toks/s]
Processed prompts:   8%|▊         | 752/10000 [00:53<12:00, 12.84it/s, est. speed input: 3669.04 toks/s, output: 1008.75 toks/s]
Processed prompts:   8%|▊         | 755/10000 [00:53<11:08, 13.84it/s, est. speed input: 3668.76 toks/s, output: 1009.68 toks/s]
Processed prompts:   8%|▊         | 757/10000 [00:54<15:47,  9.76it/s, est. speed input: 3646.57 toks/s, output: 1004.44 toks/s]
Processed prompts:   8%|▊         | 760/10000 [00:54<18:56,  8.13it/s, est. speed input: 3623.82 toks/s, output: 999.02 toks/s] 
Processed prompts:   8%|▊         | 763/10000 [00:55<17:49,  8.64it/s, est. speed input: 3615.74 toks/s, output: 997.67 toks/s]
Processed prompts:   8%|▊         | 765/10000 [00:55<16:43,  9.21it/s, est. speed input: 3612.36 toks/s, output: 997.53 toks/s]
Processed prompts:   8%|▊         | 767/10000 [00:55<19:42,  7.81it/s, est. speed input: 3595.18 toks/s, output: 993.37 toks/s]
Processed prompts:   8%|▊         | 769/10000 [00:55<18:04,  8.51it/s, est. speed input: 3591.41 toks/s, output: 992.78 toks/s]
Processed prompts:   8%|▊         | 771/10000 [00:55<17:27,  8.81it/s, est. speed input: 3586.24 toks/s, output: 992.09 toks/s]
Processed prompts:   8%|▊         | 775/10000 [00:56<13:24, 11.47it/s, est. speed input: 3587.48 toks/s, output: 993.82 toks/s]
Processed prompts:   8%|▊         | 777/10000 [00:56<18:11,  8.45it/s, est. speed input: 3567.10 toks/s, output: 988.24 toks/s]
Processed prompts:   8%|▊         | 779/10000 [00:56<19:34,  7.85it/s, est. speed input: 3555.11 toks/s, output: 985.27 toks/s]
Processed prompts:   8%|▊         | 781/10000 [00:57<17:54,  8.58it/s, est. speed input: 3551.31 toks/s, output: 984.69 toks/s]
Processed prompts:   8%|▊         | 783/10000 [00:57<17:16,  8.89it/s, est. speed input: 3546.33 toks/s, output: 983.60 toks/s]
Processed prompts:   8%|▊         | 784/10000 [00:57<19:48,  7.76it/s, est. speed input: 3536.65 toks/s, output: 981.15 toks/s]
Processed prompts:   8%|▊         | 785/10000 [00:57<20:57,  7.33it/s, est. speed input: 3529.83 toks/s, output: 979.53 toks/s]
Processed prompts:   8%|▊         | 789/10000 [00:57<13:25, 11.44it/s, est. speed input: 3533.88 toks/s, output: 981.04 toks/s]
Processed prompts:   8%|▊         | 792/10000 [00:58<13:33, 11.32it/s, est. speed input: 3527.93 toks/s, output: 979.33 toks/s]
Processed prompts:   8%|▊         | 794/10000 [00:58<14:49, 10.35it/s, est. speed input: 3520.33 toks/s, output: 977.40 toks/s]
Processed prompts:   8%|▊         | 796/10000 [00:58<14:23, 10.66it/s, est. speed input: 3517.01 toks/s, output: 976.77 toks/s]
Processed prompts:   8%|▊         | 801/10000 [00:59<15:41,  9.77it/s, est. speed input: 3501.82 toks/s, output: 973.01 toks/s]
Processed prompts:   8%|▊         | 808/10000 [00:59<11:35, 13.22it/s, est. speed input: 3506.96 toks/s, output: 975.83 toks/s]
Processed prompts:   8%|▊         | 811/10000 [00:59<12:02, 12.71it/s, est. speed input: 3502.03 toks/s, output: 974.64 toks/s]
Processed prompts:   8%|▊         | 816/10000 [00:59<10:32, 14.51it/s, est. speed input: 3504.07 toks/s, output: 975.92 toks/s]
Processed prompts:   8%|▊         | 820/10000 [01:00<10:54, 14.02it/s, est. speed input: 3499.88 toks/s, output: 975.53 toks/s]
Processed prompts:   8%|▊         | 824/10000 [01:00<10:42, 14.29it/s, est. speed input: 3498.31 toks/s, output: 975.99 toks/s]
Processed prompts:   8%|▊         | 829/10000 [01:00<09:51, 15.52it/s, est. speed input: 3499.94 toks/s, output: 977.05 toks/s]
Processed prompts:   8%|▊         | 836/10000 [01:01<08:37, 17.72it/s, est. speed input: 3505.88 toks/s, output: 979.60 toks/s]
Processed prompts:   8%|▊         | 841/10000 [01:01<09:29, 16.09it/s, est. speed input: 3501.60 toks/s, output: 978.75 toks/s]
Processed prompts:   8%|▊         | 846/10000 [01:01<09:27, 16.14it/s, est. speed input: 3501.34 toks/s, output: 979.49 toks/s]
Processed prompts:   9%|▊         | 853/10000 [01:02<08:25, 18.08it/s, est. speed input: 3507.66 toks/s, output: 982.43 toks/s]
Processed prompts:   9%|▊         | 858/10000 [01:02<09:15, 16.45it/s, est. speed input: 3503.31 toks/s, output: 982.71 toks/s]
Processed prompts:   9%|▊         | 868/10000 [01:02<07:15, 20.98it/s, est. speed input: 3519.16 toks/s, output: 987.84 toks/s]
Processed prompts:   9%|▊         | 874/10000 [01:03<08:24, 18.07it/s, est. speed input: 3513.75 toks/s, output: 987.28 toks/s]
Processed prompts:   9%|▉         | 879/10000 [01:03<08:43, 17.43it/s, est. speed input: 3513.37 toks/s, output: 987.97 toks/s]
Processed prompts:   9%|▉         | 884/10000 [01:03<08:54, 17.04it/s, est. speed input: 3512.65 toks/s, output: 988.53 toks/s]
Processed prompts:   9%|▉         | 886/10000 [01:04<10:45, 14.11it/s, est. speed input: 3502.29 toks/s, output: 986.18 toks/s]
Processed prompts:   9%|▉         | 890/10000 [01:04<09:37, 15.78it/s, est. speed input: 3506.33 toks/s, output: 988.00 toks/s]
Processed prompts:   9%|▉         | 893/10000 [01:04<10:57, 13.85it/s, est. speed input: 3499.21 toks/s, output: 986.75 toks/s]
Processed prompts:   9%|▉         | 895/10000 [01:04<12:20, 12.29it/s, est. speed input: 3491.99 toks/s, output: 985.20 toks/s]
Processed prompts:   9%|▉         | 901/10000 [01:05<09:17, 16.33it/s, est. speed input: 3500.25 toks/s, output: 988.34 toks/s]
Processed prompts:   9%|▉         | 913/10000 [01:05<06:27, 23.47it/s, est. speed input: 3521.55 toks/s, output: 995.87 toks/s]
Processed prompts:   9%|▉         | 920/10000 [01:05<08:02, 18.80it/s, est. speed input: 3515.13 toks/s, output: 995.20 toks/s]
Processed prompts:   9%|▉         | 923/10000 [01:06<09:46, 15.49it/s, est. speed input: 3505.12 toks/s, output: 993.16 toks/s]
Processed prompts:   9%|▉         | 927/10000 [01:06<09:39, 15.66it/s, est. speed input: 3505.18 toks/s, output: 993.70 toks/s]
Processed prompts:   9%|▉         | 931/10000 [01:06<09:46, 15.47it/s, est. speed input: 3504.02 toks/s, output: 993.90 toks/s]
Processed prompts:   9%|▉         | 937/10000 [01:07<08:42, 17.34it/s, est. speed input: 3509.07 toks/s, output: 995.96 toks/s]
Processed prompts:   9%|▉         | 942/10000 [01:07<09:04, 16.65it/s, est. speed input: 3507.70 toks/s, output: 996.44 toks/s]
Processed prompts:   9%|▉         | 945/10000 [01:07<10:22, 14.54it/s, est. speed input: 3501.35 toks/s, output: 995.21 toks/s]
Processed prompts:   9%|▉         | 947/10000 [01:08<11:44, 12.85it/s, est. speed input: 3494.80 toks/s, output: 993.91 toks/s]
Processed prompts:  10%|▉         | 952/10000 [01:08<09:44, 15.49it/s, est. speed input: 3499.71 toks/s, output: 996.58 toks/s]
Processed prompts:  10%|▉         | 955/10000 [01:08<11:03, 13.63it/s, est. speed input: 3493.92 toks/s, output: 995.80 toks/s]
Processed prompts:  10%|▉         | 961/10000 [01:08<09:07, 16.52it/s, est. speed input: 3499.13 toks/s, output: 998.83 toks/s]
Processed prompts:  10%|▉         | 966/10000 [01:09<10:33, 14.26it/s, est. speed input: 3491.28 toks/s, output: 997.41 toks/s]
Processed prompts:  10%|▉         | 973/10000 [01:09<08:02, 18.70it/s, est. speed input: 3502.79 toks/s, output: 1002.57 toks/s]
Processed prompts:  10%|▉         | 977/10000 [01:09<08:53, 16.91it/s, est. speed input: 3499.55 toks/s, output: 1002.45 toks/s]
Processed prompts:  10%|▉         | 980/10000 [01:09<09:51, 15.25it/s, est. speed input: 3495.01 toks/s, output: 1001.82 toks/s]
Processed prompts:  10%|▉         | 982/10000 [01:10<11:23, 13.19it/s, est. speed input: 3488.47 toks/s, output: 1000.47 toks/s]
Processed prompts:  10%|▉         | 988/10000 [01:10<08:49, 17.03it/s, est. speed input: 3496.33 toks/s, output: 1003.50 toks/s]
Processed prompts:  10%|▉         | 991/10000 [01:10<10:31, 14.27it/s, est. speed input: 3489.43 toks/s, output: 1001.93 toks/s]
Processed prompts:  10%|▉         | 996/10000 [01:11<09:23, 15.98it/s, est. speed input: 3492.41 toks/s, output: 1003.32 toks/s]
Processed prompts:  10%|█         | 1000/10000 [01:11<09:37, 15.59it/s, est. speed input: 3491.15 toks/s, output: 1003.63 toks/s]
Processed prompts:  10%|█         | 1002/10000 [01:11<11:18, 13.26it/s, est. speed input: 3484.11 toks/s, output: 1002.28 toks/s]
Processed prompts:  10%|█         | 1004/10000 [01:11<11:36, 12.92it/s, est. speed input: 3481.80 toks/s, output: 1002.04 toks/s]
Processed prompts:  10%|█         | 1006/10000 [01:11<12:32, 11.95it/s, est. speed input: 3477.08 toks/s, output: 1001.07 toks/s]
Processed prompts:  10%|█         | 1008/10000 [01:12<12:38, 11.86it/s, est. speed input: 3474.82 toks/s, output: 1001.07 toks/s]
Processed prompts:  10%|█         | 1013/10000 [01:12<09:43, 15.41it/s, est. speed input: 3479.44 toks/s, output: 1003.54 toks/s]
Processed prompts:  10%|█         | 1015/10000 [01:12<12:48, 11.69it/s, est. speed input: 3469.51 toks/s, output: 1001.02 toks/s]
Processed prompts:  10%|█         | 1017/10000 [01:12<13:15, 11.30it/s, est. speed input: 3465.67 toks/s, output: 1000.02 toks/s]
Processed prompts:  10%|█         | 1019/10000 [01:13<18:31,  8.08it/s, est. speed input: 3449.59 toks/s, output: 995.25 toks/s] 
Processed prompts:  10%|█         | 1020/10000 [01:13<19:35,  7.64it/s, est. speed input: 3444.47 toks/s, output: 993.89 toks/s]
Processed prompts:  10%|█         | 1023/10000 [01:13<15:14,  9.81it/s, est. speed input: 3445.24 toks/s, output: 994.55 toks/s]
Processed prompts:  10%|█         | 1028/10000 [01:13<10:59, 13.60it/s, est. speed input: 3449.28 toks/s, output: 996.65 toks/s]
Processed prompts:  10%|█         | 1032/10000 [01:14<10:35, 14.12it/s, est. speed input: 3448.23 toks/s, output: 996.64 toks/s]
Processed prompts:  10%|█         | 1034/10000 [01:14<14:55, 10.01it/s, est. speed input: 3433.89 toks/s, output: 992.67 toks/s]
Processed prompts:  10%|█         | 1036/10000 [01:14<14:29, 10.31it/s, est. speed input: 3431.60 toks/s, output: 992.61 toks/s]
Processed prompts:  10%|█         | 1038/10000 [01:14<14:06, 10.59it/s, est. speed input: 3429.31 toks/s, output: 992.20 toks/s]
Processed prompts:  10%|█         | 1042/10000 [01:15<12:03, 12.39it/s, est. speed input: 3429.24 toks/s, output: 992.27 toks/s]
Processed prompts:  10%|█         | 1044/10000 [01:15<13:40, 10.91it/s, est. speed input: 3422.90 toks/s, output: 990.68 toks/s]
Processed prompts:  10%|█         | 1047/10000 [01:15<12:20, 12.09it/s, est. speed input: 3422.33 toks/s, output: 990.78 toks/s]
Processed prompts:  10%|█         | 1050/10000 [01:15<12:17, 12.14it/s, est. speed input: 3419.75 toks/s, output: 990.20 toks/s]
Processed prompts:  11%|█         | 1054/10000 [01:16<11:08, 13.38it/s, est. speed input: 3419.70 toks/s, output: 990.49 toks/s]
Processed prompts:  11%|█         | 1058/10000 [01:16<10:39, 13.99it/s, est. speed input: 3418.77 toks/s, output: 990.53 toks/s]
Processed prompts:  11%|█         | 1060/10000 [01:16<12:25, 11.99it/s, est. speed input: 3412.57 toks/s, output: 988.99 toks/s]
Processed prompts:  11%|█         | 1065/10000 [01:16<09:20, 15.93it/s, est. speed input: 3418.70 toks/s, output: 990.75 toks/s]
Processed prompts:  11%|█         | 1067/10000 [01:17<11:57, 12.44it/s, est. speed input: 3410.79 toks/s, output: 988.35 toks/s]
Processed prompts:  11%|█         | 1069/10000 [01:17<12:10, 12.22it/s, est. speed input: 3408.57 toks/s, output: 988.01 toks/s]
Processed prompts:  11%|█         | 1072/10000 [01:17<11:19, 13.13it/s, est. speed input: 3408.52 toks/s, output: 988.15 toks/s]
Processed prompts:  11%|█         | 1078/10000 [01:17<09:01, 16.47it/s, est. speed input: 3413.16 toks/s, output: 989.86 toks/s]
Processed prompts:  11%|█         | 1080/10000 [01:18<11:35, 12.82it/s, est. speed input: 3405.24 toks/s, output: 987.56 toks/s]
Processed prompts:  11%|█         | 1084/10000 [01:18<10:10, 14.59it/s, est. speed input: 3407.31 toks/s, output: 988.23 toks/s]
Processed prompts:  11%|█         | 1086/10000 [01:18<11:50, 12.55it/s, est. speed input: 3401.96 toks/s, output: 986.91 toks/s]
Processed prompts:  11%|█         | 1090/10000 [01:18<10:20, 14.36it/s, est. speed input: 3403.74 toks/s, output: 987.80 toks/s]
Processed prompts:  11%|█         | 1096/10000 [01:18<08:34, 17.30it/s, est. speed input: 3408.69 toks/s, output: 989.80 toks/s]
Processed prompts:  11%|█         | 1102/10000 [01:19<08:01, 18.48it/s, est. speed input: 3412.46 toks/s, output: 991.58 toks/s]
Processed prompts:  11%|█         | 1109/10000 [01:19<07:28, 19.81it/s, est. speed input: 3418.22 toks/s, output: 993.66 toks/s]
Processed prompts:  11%|█         | 1111/10000 [01:19<09:53, 14.98it/s, est. speed input: 3408.97 toks/s, output: 991.30 toks/s]
Processed prompts:  11%|█         | 1116/10000 [01:20<08:30, 17.40it/s, est. speed input: 3413.99 toks/s, output: 992.95 toks/s]
Processed prompts:  11%|█▏        | 1125/10000 [01:20<06:40, 22.17it/s, est. speed input: 3425.77 toks/s, output: 997.35 toks/s]
Processed prompts:  11%|█▏        | 1130/10000 [01:20<08:08, 18.17it/s, est. speed input: 3421.47 toks/s, output: 996.36 toks/s]
Processed prompts:  11%|█▏        | 1134/10000 [01:21<08:43, 16.93it/s, est. speed input: 3419.76 toks/s, output: 996.35 toks/s]
Processed prompts:  11%|█▏        | 1141/10000 [01:21<07:32, 19.56it/s, est. speed input: 3426.84 toks/s, output: 998.83 toks/s]
Processed prompts:  11%|█▏        | 1146/10000 [01:21<08:10, 18.04it/s, est. speed input: 3425.83 toks/s, output: 999.08 toks/s]
Processed prompts:  12%|█▏        | 1152/10000 [01:21<07:48, 18.88it/s, est. speed input: 3429.50 toks/s, output: 1000.59 toks/s]
Processed prompts:  12%|█▏        | 1154/10000 [01:22<09:37, 15.33it/s, est. speed input: 3422.47 toks/s, output: 998.62 toks/s] 
Processed prompts:  12%|█▏        | 1157/10000 [01:22<09:20, 15.78it/s, est. speed input: 3422.88 toks/s, output: 998.95 toks/s]
Processed prompts:  12%|█▏        | 1163/10000 [01:22<08:06, 18.17it/s, est. speed input: 3427.40 toks/s, output: 1000.64 toks/s]
Processed prompts:  12%|█▏        | 1166/10000 [01:22<09:37, 15.30it/s, est. speed input: 3422.34 toks/s, output: 999.26 toks/s] 
Processed prompts:  12%|█▏        | 1170/10000 [01:23<09:26, 15.58it/s, est. speed input: 3422.56 toks/s, output: 999.46 toks/s]
Processed prompts:  12%|█▏        | 1175/10000 [01:23<08:47, 16.74it/s, est. speed input: 3424.62 toks/s, output: 1000.52 toks/s]
Processed prompts:  12%|█▏        | 1180/10000 [01:23<08:30, 17.28it/s, est. speed input: 3426.23 toks/s, output: 1001.58 toks/s]
Processed prompts:  12%|█▏        | 1185/10000 [01:24<08:27, 17.37it/s, est. speed input: 3427.13 toks/s, output: 1002.60 toks/s]
Processed prompts:  12%|█▏        | 1188/10000 [01:24<09:35, 15.32it/s, est. speed input: 3423.43 toks/s, output: 1002.06 toks/s]
Processed prompts:  12%|█▏        | 1193/10000 [01:24<08:41, 16.88it/s, est. speed input: 3425.94 toks/s, output: 1003.20 toks/s]
Processed prompts:  12%|█▏        | 1198/10000 [01:24<08:25, 17.42it/s, est. speed input: 3427.77 toks/s, output: 1004.03 toks/s]
Processed prompts:  12%|█▏        | 1200/10000 [01:25<10:09, 14.44it/s, est. speed input: 3421.95 toks/s, output: 1002.60 toks/s]
Processed prompts:  12%|█▏        | 1202/10000 [01:25<10:30, 13.96it/s, est. speed input: 3420.40 toks/s, output: 1002.46 toks/s]
Processed prompts:  12%|█▏        | 1207/10000 [01:25<09:42, 15.10it/s, est. speed input: 3421.35 toks/s, output: 1003.39 toks/s]
Processed prompts:  12%|█▏        | 1210/10000 [01:25<09:59, 14.67it/s, est. speed input: 3419.89 toks/s, output: 1003.38 toks/s]
Processed prompts:  12%|█▏        | 1214/10000 [01:26<12:17, 11.91it/s, est. speed input: 3410.43 toks/s, output: 1000.99 toks/s]
Processed prompts:  12%|█▏        | 1220/10000 [01:26<11:14, 13.02it/s, est. speed input: 3412.99 toks/s, output: 1001.26 toks/s]
Processed prompts:  12%|█▏        | 1222/10000 [01:27<15:27,  9.46it/s, est. speed input: 3394.50 toks/s, output: 996.91 toks/s] 
Processed prompts:  12%|█▏        | 1224/10000 [01:27<20:47,  7.03it/s, est. speed input: 3376.12 toks/s, output: 992.05 toks/s]
Processed prompts:  12%|█▏        | 1227/10000 [01:28<21:50,  6.69it/s, est. speed input: 3363.72 toks/s, output: 988.81 toks/s]
Processed prompts:  12%|█▏        | 1231/10000 [01:29<24:26,  5.98it/s, est. speed input: 3341.97 toks/s, output: 983.15 toks/s]
Processed prompts:  12%|█▏        | 1232/10000 [01:29<31:32,  4.63it/s, est. speed input: 3321.83 toks/s, output: 977.42 toks/s]
Processed prompts:  12%|█▏        | 1233/10000 [01:30<41:21,  3.53it/s, est. speed input: 3298.29 toks/s, output: 970.64 toks/s]
Processed prompts:  12%|█▏        | 1235/10000 [01:30<41:58,  3.48it/s, est. speed input: 3281.46 toks/s, output: 966.01 toks/s]
Processed prompts:  12%|█▏        | 1239/10000 [01:31<36:18,  4.02it/s, est. speed input: 3261.86 toks/s, output: 960.60 toks/s]
Processed prompts:  12%|█▏        | 1240/10000 [01:32<41:24,  3.53it/s, est. speed input: 3246.24 toks/s, output: 956.18 toks/s]
Processed prompts:  12%|█▏        | 1244/10000 [01:32<34:21,  4.25it/s, est. speed input: 3230.72 toks/s, output: 952.08 toks/s]
Processed prompts:  12%|█▏        | 1247/10000 [01:33<32:46,  4.45it/s, est. speed input: 3216.41 toks/s, output: 948.50 toks/s]
Processed prompts:  12%|█▏        | 1249/10000 [01:34<35:20,  4.13it/s, est. speed input: 3200.40 toks/s, output: 944.22 toks/s]
Processed prompts:  13%|█▎        | 1251/10000 [01:34<37:23,  3.90it/s, est. speed input: 3184.27 toks/s, output: 939.82 toks/s]
Processed prompts:  13%|█▎        | 1255/10000 [01:35<29:20,  4.97it/s, est. speed input: 3177.19 toks/s, output: 938.08 toks/s]
Processed prompts:  13%|█▎        | 1256/10000 [01:36<40:38,  3.59it/s, est. speed input: 3152.70 toks/s, output: 931.09 toks/s]
Processed prompts:  13%|█▎        | 1258/10000 [01:36<43:30,  3.35it/s, est. speed input: 3134.51 toks/s, output: 925.84 toks/s]
Processed prompts:  13%|█▎        | 1262/10000 [01:37<32:30,  4.48it/s, est. speed input: 3125.67 toks/s, output: 923.97 toks/s]
Processed prompts:  13%|█▎        | 1263/10000 [01:37<37:53,  3.84it/s, est. speed input: 3111.94 toks/s, output: 920.19 toks/s]
Processed prompts:  13%|█▎        | 1266/10000 [01:38<34:40,  4.20it/s, est. speed input: 3098.80 toks/s, output: 917.05 toks/s]
Processed prompts:  13%|█▎        | 1268/10000 [01:38<31:29,  4.62it/s, est. speed input: 3092.59 toks/s, output: 915.76 toks/s]
Processed prompts:  13%|█▎        | 1271/10000 [01:39<27:11,  5.35it/s, est. speed input: 3085.84 toks/s, output: 913.98 toks/s]
Processed prompts:  13%|█▎        | 1272/10000 [01:39<35:43,  4.07it/s, est. speed input: 3069.35 toks/s, output: 909.22 toks/s]
Processed prompts:  13%|█▎        | 1275/10000 [01:40<29:33,  4.92it/s, est. speed input: 3062.70 toks/s, output: 907.57 toks/s]
Processed prompts:  13%|█▎        | 1279/10000 [01:40<24:40,  5.89it/s, est. speed input: 3055.17 toks/s, output: 906.47 toks/s]
Processed prompts:  13%|█▎        | 1282/10000 [01:41<24:42,  5.88it/s, est. speed input: 3045.10 toks/s, output: 903.80 toks/s]
Processed prompts:  13%|█▎        | 1285/10000 [01:41<25:51,  5.62it/s, est. speed input: 3032.56 toks/s, output: 900.70 toks/s]
Processed prompts:  13%|█▎        | 1287/10000 [01:42<31:22,  4.63it/s, est. speed input: 3017.98 toks/s, output: 896.06 toks/s]
Processed prompts:  13%|█▎        | 1291/10000 [01:43<30:36,  4.74it/s, est. speed input: 2999.63 toks/s, output: 891.87 toks/s]
Processed prompts:  13%|█▎        | 1292/10000 [01:43<35:34,  4.08it/s, est. speed input: 2989.38 toks/s, output: 888.50 toks/s]
Processed prompts:  13%|█▎        | 1295/10000 [01:44<33:23,  4.34it/s, est. speed input: 2974.70 toks/s, output: 885.33 toks/s]
Processed prompts:  13%|█▎        | 1297/10000 [01:44<34:04,  4.26it/s, est. speed input: 2963.66 toks/s, output: 882.64 toks/s]
Processed prompts:  13%|█▎        | 1299/10000 [01:45<31:31,  4.60it/s, est. speed input: 2957.15 toks/s, output: 881.18 toks/s]
Processed prompts:  13%|█▎        | 1301/10000 [01:45<25:59,  5.58it/s, est. speed input: 2956.03 toks/s, output: 881.53 toks/s]
Processed prompts:  13%|█▎        | 1305/10000 [01:45<21:41,  6.68it/s, est. speed input: 2949.97 toks/s, output: 880.45 toks/s]
Processed prompts:  13%|█▎        | 1308/10000 [01:46<22:58,  6.30it/s, est. speed input: 2940.42 toks/s, output: 878.06 toks/s]
Processed prompts:  13%|█▎        | 1314/10000 [01:46<18:24,  7.86it/s, est. speed input: 2934.81 toks/s, output: 877.60 toks/s]
Processed prompts:  13%|█▎        | 1320/10000 [01:47<20:57,  6.90it/s, est. speed input: 2916.27 toks/s, output: 873.00 toks/s]
Processed prompts:  13%|█▎        | 1327/10000 [01:48<20:56,  6.91it/s, est. speed input: 2901.48 toks/s, output: 869.14 toks/s]
Processed prompts:  13%|█▎        | 1330/10000 [01:49<26:37,  5.43it/s, est. speed input: 2879.14 toks/s, output: 863.09 toks/s]
Processed prompts:  13%|█▎        | 1332/10000 [01:50<29:10,  4.95it/s, est. speed input: 2867.04 toks/s, output: 859.87 toks/s]
Processed prompts:  13%|█▎        | 1339/10000 [01:50<19:41,  7.33it/s, est. speed input: 2869.20 toks/s, output: 861.70 toks/s]
Processed prompts:  13%|█▎        | 1347/10000 [01:51<18:18,  7.87it/s, est. speed input: 2858.75 toks/s, output: 859.81 toks/s]
Processed prompts:  13%|█▎        | 1349/10000 [01:52<27:19,  5.28it/s, est. speed input: 2831.49 toks/s, output: 851.89 toks/s]
Processed prompts:  14%|█▎        | 1351/10000 [01:53<28:01,  5.14it/s, est. speed input: 2823.87 toks/s, output: 849.91 toks/s]
Processed prompts:  14%|█▎        | 1355/10000 [01:53<23:47,  6.06it/s, est. speed input: 2820.51 toks/s, output: 849.56 toks/s]
Processed prompts:  14%|█▎        | 1359/10000 [01:54<22:25,  6.42it/s, est. speed input: 2813.50 toks/s, output: 848.09 toks/s]
Processed prompts:  14%|█▎        | 1367/10000 [01:54<17:48,  8.08it/s, est. speed input: 2808.35 toks/s, output: 847.76 toks/s]
Processed prompts:  14%|█▎        | 1371/10000 [01:55<21:53,  6.57it/s, est. speed input: 2791.92 toks/s, output: 843.26 toks/s]
Processed prompts:  14%|█▍        | 1375/10000 [01:56<23:15,  6.18it/s, est. speed input: 2779.74 toks/s, output: 840.19 toks/s]
Processed prompts:  14%|█▍        | 1378/10000 [01:57<22:52,  6.28it/s, est. speed input: 2773.03 toks/s, output: 838.85 toks/s]
Processed prompts:  14%|█▍        | 1384/10000 [01:57<18:20,  7.83it/s, est. speed input: 2771.09 toks/s, output: 839.34 toks/s]
Processed prompts:  14%|█▍        | 1386/10000 [01:58<26:18,  5.46it/s, est. speed input: 2751.35 toks/s, output: 833.64 toks/s]
Processed prompts:  14%|█▍        | 1389/10000 [01:59<27:09,  5.29it/s, est. speed input: 2741.49 toks/s, output: 830.96 toks/s]
Processed prompts:  14%|█▍        | 1394/10000 [01:59<23:15,  6.17it/s, est. speed input: 2736.13 toks/s, output: 829.90 toks/s]
Processed prompts:  14%|█▍        | 1395/10000 [02:00<28:57,  4.95it/s, est. speed input: 2724.67 toks/s, output: 826.68 toks/s]
Processed prompts:  14%|█▍        | 1400/10000 [02:00<22:19,  6.42it/s, est. speed input: 2721.88 toks/s, output: 826.62 toks/s]
Processed prompts:  14%|█▍        | 1404/10000 [02:01<21:31,  6.66it/s, est. speed input: 2716.91 toks/s, output: 825.55 toks/s]
Processed prompts:  14%|█▍        | 1406/10000 [02:02<27:50,  5.14it/s, est. speed input: 2702.49 toks/s, output: 821.59 toks/s]
Processed prompts:  14%|█▍        | 1408/10000 [02:02<29:11,  4.90it/s, est. speed input: 2694.60 toks/s, output: 819.76 toks/s]
Processed prompts:  14%|█▍        | 1411/10000 [02:02<23:45,  6.02it/s, est. speed input: 2693.54 toks/s, output: 820.11 toks/s]
Processed prompts:  14%|█▍        | 1418/10000 [02:03<15:54,  8.99it/s, est. speed input: 2694.85 toks/s, output: 821.59 toks/s]
Processed prompts:  14%|█▍        | 1420/10000 [02:04<25:08,  5.69it/s, est. speed input: 2676.35 toks/s, output: 816.50 toks/s]
Processed prompts:  14%|█▍        | 1422/10000 [02:04<26:40,  5.36it/s, est. speed input: 2669.25 toks/s, output: 814.72 toks/s]
Processed prompts:  14%|█▍        | 1425/10000 [02:05<24:44,  5.78it/s, est. speed input: 2665.36 toks/s, output: 813.92 toks/s]
Processed prompts:  14%|█▍        | 1430/10000 [02:05<20:03,  7.12it/s, est. speed input: 2662.54 toks/s, output: 814.27 toks/s]
Processed prompts:  14%|█▍        | 1437/10000 [02:06<16:28,  8.66it/s, est. speed input: 2660.23 toks/s, output: 814.31 toks/s]
Processed prompts:  14%|█▍        | 1443/10000 [02:07<20:00,  7.13it/s, est. speed input: 2645.86 toks/s, output: 810.82 toks/s]
Processed prompts:  14%|█▍        | 1448/10000 [02:08<21:23,  6.67it/s, est. speed input: 2634.48 toks/s, output: 808.56 toks/s]
Processed prompts:  15%|█▍        | 1452/10000 [02:09<24:50,  5.74it/s, est. speed input: 2620.34 toks/s, output: 804.95 toks/s]
Processed prompts:  15%|█▍        | 1456/10000 [02:09<23:59,  5.94it/s, est. speed input: 2614.23 toks/s, output: 803.70 toks/s]
Processed prompts:  15%|█▍        | 1462/10000 [02:10<21:11,  6.72it/s, est. speed input: 2609.52 toks/s, output: 802.94 toks/s]
Processed prompts:  15%|█▍        | 1464/10000 [02:11<26:38,  5.34it/s, est. speed input: 2596.46 toks/s, output: 799.12 toks/s]
Processed prompts:  15%|█▍        | 1468/10000 [02:11<24:22,  5.83it/s, est. speed input: 2591.20 toks/s, output: 798.19 toks/s]
Processed prompts:  15%|█▍        | 1472/10000 [02:13<31:45,  4.47it/s, est. speed input: 2569.30 toks/s, output: 792.40 toks/s]
Processed prompts:  15%|█▍        | 1475/10000 [02:13<32:37,  4.35it/s, est. speed input: 2559.34 toks/s, output: 789.72 toks/s]
Processed prompts:  15%|█▍        | 1478/10000 [02:14<27:30,  5.16it/s, est. speed input: 2558.44 toks/s, output: 789.85 toks/s]
Processed prompts:  15%|█▍        | 1479/10000 [02:14<30:44,  4.62it/s, est. speed input: 2552.29 toks/s, output: 788.11 toks/s]
Processed prompts:  15%|█▍        | 1480/10000 [02:14<33:14,  4.27it/s, est. speed input: 2547.24 toks/s, output: 786.79 toks/s]
Processed prompts:  15%|█▍        | 1483/10000 [02:15<26:28,  5.36it/s, est. speed input: 2545.13 toks/s, output: 786.86 toks/s]
Processed prompts:  15%|█▍        | 1485/10000 [02:15<30:07,  4.71it/s, est. speed input: 2542.42 toks/s, output: 784.67 toks/s]
Processed prompts:  15%|█▍        | 1486/10000 [02:16<39:40,  3.58it/s, est. speed input: 2531.66 toks/s, output: 781.67 toks/s]
Processed prompts:  15%|█▍        | 1488/10000 [02:16<35:51,  3.96it/s, est. speed input: 2527.23 toks/s, output: 780.81 toks/s]
Processed prompts:  15%|█▍        | 1491/10000 [02:16<23:32,  6.02it/s, est. speed input: 2530.62 toks/s, output: 781.96 toks/s]
Processed prompts:  15%|█▍        | 1494/10000 [02:17<20:34,  6.89it/s, est. speed input: 2537.83 toks/s, output: 781.87 toks/s]
Processed prompts:  15%|█▍        | 1496/10000 [02:17<21:12,  6.68it/s, est. speed input: 2540.30 toks/s, output: 781.11 toks/s]
Processed prompts:  15%|█▍        | 1497/10000 [02:17<24:00,  5.90it/s, est. speed input: 2536.49 toks/s, output: 780.24 toks/s]
Processed prompts:  15%|█▍        | 1499/10000 [02:18<21:40,  6.54it/s, est. speed input: 2542.84 toks/s, output: 779.97 toks/s]
Processed prompts:  15%|█▌        | 1500/10000 [02:18<24:46,  5.72it/s, est. speed input: 2539.12 toks/s, output: 779.06 toks/s]
Processed prompts:  15%|█▌        | 1501/10000 [02:18<26:33,  5.33it/s, est. speed input: 2536.23 toks/s, output: 778.47 toks/s]
Processed prompts:  15%|█▌        | 1502/10000 [02:18<27:16,  5.19it/s, est. speed input: 2533.81 toks/s, output: 778.04 toks/s]
Processed prompts:  15%|█▌        | 1506/10000 [02:19<16:29,  8.58it/s, est. speed input: 2558.07 toks/s, output: 778.99 toks/s]
Processed prompts:  15%|█▌        | 1510/10000 [02:19<15:16,  9.26it/s, est. speed input: 2568.90 toks/s, output: 779.09 toks/s]
Processed prompts:  15%|█▌        | 1511/10000 [02:19<20:03,  7.05it/s, est. speed input: 2565.60 toks/s, output: 777.69 toks/s]
Processed prompts:  15%|█▌        | 1513/10000 [02:20<24:39,  5.74it/s, est. speed input: 2569.25 toks/s, output: 775.90 toks/s]
Processed prompts:  15%|█▌        | 1516/10000 [02:20<21:12,  6.67it/s, est. speed input: 2586.29 toks/s, output: 775.84 toks/s]
Processed prompts:  15%|█▌        | 1518/10000 [02:20<20:51,  6.78it/s, est. speed input: 2592.23 toks/s, output: 775.47 toks/s]
Processed prompts:  15%|█▌        | 1519/10000 [02:21<23:52,  5.92it/s, est. speed input: 2592.27 toks/s, output: 774.55 toks/s]
Processed prompts:  15%|█▌        | 1522/10000 [02:21<18:53,  7.48it/s, est. speed input: 2598.49 toks/s, output: 775.05 toks/s]
Processed prompts:  15%|█▌        | 1525/10000 [02:22<23:33,  6.00it/s, est. speed input: 2606.25 toks/s, output: 773.16 toks/s]
Processed prompts:  15%|█▌        | 1528/10000 [02:22<17:11,  8.21it/s, est. speed input: 2620.56 toks/s, output: 774.29 toks/s]
Processed prompts:  15%|█▌        | 1531/10000 [02:22<16:41,  8.46it/s, est. speed input: 2630.82 toks/s, output: 774.36 toks/s]
Processed prompts:  15%|█▌        | 1533/10000 [02:22<18:08,  7.78it/s, est. speed input: 2632.16 toks/s, output: 773.63 toks/s]
Processed prompts:  15%|█▌        | 1535/10000 [02:23<18:34,  7.60it/s, est. speed input: 2641.37 toks/s, output: 773.24 toks/s]
Processed prompts:  15%|█▌        | 1536/10000 [02:23<26:32,  5.31it/s, est. speed input: 2637.85 toks/s, output: 771.15 toks/s]
Processed prompts:  15%|█▌        | 1541/10000 [02:24<19:55,  7.08it/s, est. speed input: 2655.02 toks/s, output: 771.34 toks/s]
Processed prompts:  15%|█▌        | 1545/10000 [02:24<15:28,  9.10it/s, est. speed input: 2664.53 toks/s, output: 772.45 toks/s]
Processed prompts:  16%|█▌        | 1551/10000 [02:24<12:40, 11.11it/s, est. speed input: 2691.40 toks/s, output: 773.80 toks/s]
Processed prompts:  16%|█▌        | 1555/10000 [02:25<13:55, 10.11it/s, est. speed input: 2705.72 toks/s, output: 773.66 toks/s]
Processed prompts:  16%|█▌        | 1561/10000 [02:25<11:47, 11.93it/s, est. speed input: 2738.81 toks/s, output: 774.99 toks/s]
Processed prompts:  16%|█▌        | 1564/10000 [02:26<13:59, 10.05it/s, est. speed input: 2741.08 toks/s, output: 774.09 toks/s]
Processed prompts:  16%|█▌        | 1567/10000 [02:26<14:24,  9.75it/s, est. speed input: 2750.41 toks/s, output: 773.95 toks/s]
Processed prompts:  16%|█▌        | 1569/10000 [02:26<15:56,  8.81it/s, est. speed input: 2755.99 toks/s, output: 773.48 toks/s]
Processed prompts:  16%|█▌        | 1572/10000 [02:27<15:04,  9.32it/s, est. speed input: 2761.72 toks/s, output: 773.76 toks/s]
Processed prompts:  16%|█▌        | 1576/10000 [02:27<13:48, 10.16it/s, est. speed input: 2772.79 toks/s, output: 774.29 toks/s]
Processed prompts:  16%|█▌        | 1580/10000 [02:28<19:23,  7.24it/s, est. speed input: 2776.84 toks/s, output: 771.99 toks/s]
Processed prompts:  16%|█▌        | 1584/10000 [02:28<14:20,  9.78it/s, est. speed input: 2793.68 toks/s, output: 773.81 toks/s]
Processed prompts:  16%|█▌        | 1590/10000 [02:28<11:51, 11.83it/s, est. speed input: 2820.93 toks/s, output: 775.35 toks/s]
Processed prompts:  16%|█▌        | 1595/10000 [02:29<12:42, 11.02it/s, est. speed input: 2850.27 toks/s, output: 775.42 toks/s]
Processed prompts:  16%|█▌        | 1598/10000 [02:29<14:33,  9.62it/s, est. speed input: 2853.48 toks/s, output: 774.80 toks/s]
Processed prompts:  16%|█▌        | 1602/10000 [02:30<13:39, 10.25it/s, est. speed input: 2870.93 toks/s, output: 775.43 toks/s]
Processed prompts:  16%|█▌        | 1609/10000 [02:30<11:07, 12.57it/s, est. speed input: 2900.75 toks/s, output: 777.39 toks/s]
Processed prompts:  16%|█▌        | 1614/10000 [02:30<12:30, 11.18it/s, est. speed input: 2918.33 toks/s, output: 777.42 toks/s]
Processed prompts:  16%|█▌        | 1616/10000 [02:31<17:27,  8.00it/s, est. speed input: 2914.67 toks/s, output: 775.23 toks/s]
Processed prompts:  16%|█▌        | 1622/10000 [02:31<12:44, 10.96it/s, est. speed input: 2945.26 toks/s, output: 777.43 toks/s]
Processed prompts:  16%|█▌        | 1624/10000 [02:32<16:23,  8.52it/s, est. speed input: 2947.42 toks/s, output: 775.89 toks/s]
Processed prompts:  16%|█▋        | 1630/10000 [02:32<12:24, 11.24it/s, est. speed input: 2967.70 toks/s, output: 777.84 toks/s]
Processed prompts:  16%|█▋        | 1638/10000 [02:33<10:37, 13.12it/s, est. speed input: 2998.92 toks/s, output: 780.25 toks/s]
Processed prompts:  16%|█▋        | 1643/10000 [02:33<12:20, 11.29it/s, est. speed input: 3008.29 toks/s, output: 779.88 toks/s]
Processed prompts:  16%|█▋        | 1646/10000 [02:34<13:54, 10.02it/s, est. speed input: 3016.33 toks/s, output: 779.24 toks/s]
Processed prompts:  16%|█▋        | 1650/10000 [02:34<13:06, 10.62it/s, est. speed input: 3037.90 toks/s, output: 779.79 toks/s]
Processed prompts:  17%|█▋        | 1652/10000 [02:34<14:00,  9.93it/s, est. speed input: 3044.67 toks/s, output: 779.62 toks/s]
Processed prompts:  17%|█▋        | 1658/10000 [02:35<13:12, 10.53it/s, est. speed input: 3059.96 toks/s, output: 780.36 toks/s]
Processed prompts:  17%|█▋        | 1661/10000 [02:35<13:04, 10.64it/s, est. speed input: 3070.42 toks/s, output: 780.57 toks/s]
Processed prompts:  17%|█▋        | 1664/10000 [02:35<13:40, 10.16it/s, est. speed input: 3075.99 toks/s, output: 780.53 toks/s]
Processed prompts:  17%|█▋        | 1666/10000 [02:36<15:12,  9.13it/s, est. speed input: 3074.72 toks/s, output: 780.15 toks/s]
Processed prompts:  17%|█▋        | 1668/10000 [02:36<15:07,  9.18it/s, est. speed input: 3089.23 toks/s, output: 780.18 toks/s]
Processed prompts:  17%|█▋        | 1673/10000 [02:36<12:34, 11.04it/s, est. speed input: 3110.48 toks/s, output: 781.15 toks/s]
Processed prompts:  17%|█▋        | 1675/10000 [02:37<15:43,  8.83it/s, est. speed input: 3112.22 toks/s, output: 780.31 toks/s]
Processed prompts:  17%|█▋        | 1678/10000 [02:37<14:34,  9.51it/s, est. speed input: 3129.68 toks/s, output: 780.72 toks/s]
Processed prompts:  17%|█▋        | 1683/10000 [02:37<12:18, 11.26it/s, est. speed input: 3146.28 toks/s, output: 781.69 toks/s]
Processed prompts:  17%|█▋        | 1685/10000 [02:38<15:23,  9.01it/s, est. speed input: 3149.78 toks/s, output: 780.92 toks/s]
Processed prompts:  17%|█▋        | 1691/10000 [02:38<10:43, 12.92it/s, est. speed input: 3179.75 toks/s, output: 782.98 toks/s]
Processed prompts:  17%|█▋        | 1698/10000 [02:38<10:44, 12.88it/s, est. speed input: 3208.69 toks/s, output: 784.10 toks/s]
Processed prompts:  17%|█▋        | 1701/10000 [02:39<13:29, 10.25it/s, est. speed input: 3213.17 toks/s, output: 783.03 toks/s]
Processed prompts:  17%|█▋        | 1703/10000 [02:39<15:02,  9.20it/s, est. speed input: 3215.28 toks/s, output: 782.64 toks/s]
Processed prompts:  17%|█▋        | 1706/10000 [02:39<13:28, 10.26it/s, est. speed input: 3221.30 toks/s, output: 783.43 toks/s]
Processed prompts:  17%|█▋        | 1710/10000 [02:40<12:48, 10.79it/s, est. speed input: 3232.72 toks/s, output: 783.89 toks/s]
Processed prompts:  17%|█▋        | 1713/10000 [02:40<13:55,  9.91it/s, est. speed input: 3242.28 toks/s, output: 783.79 toks/s]
Processed prompts:  17%|█▋        | 1715/10000 [02:40<15:21,  8.99it/s, est. speed input: 3242.68 toks/s, output: 783.36 toks/s]
Processed prompts:  17%|█▋        | 1719/10000 [02:41<13:01, 10.60it/s, est. speed input: 3255.07 toks/s, output: 784.31 toks/s]
Processed prompts:  17%|█▋        | 1721/10000 [02:41<15:11,  9.09it/s, est. speed input: 3260.58 toks/s, output: 783.85 toks/s]
Processed prompts:  17%|█▋        | 1725/10000 [02:41<12:17, 11.22it/s, est. speed input: 3277.38 toks/s, output: 784.99 toks/s]
Processed prompts:  17%|█▋        | 1727/10000 [02:42<16:03,  8.59it/s, est. speed input: 3282.75 toks/s, output: 784.02 toks/s]
Processed prompts:  17%|█▋        | 1730/10000 [02:42<14:47,  9.32it/s, est. speed input: 3285.67 toks/s, output: 784.21 toks/s]
Processed prompts:  17%|█▋        | 1732/10000 [02:42<16:34,  8.31it/s, est. speed input: 3286.80 toks/s, output: 783.77 toks/s]
Processed prompts:  17%|█▋        | 1735/10000 [02:43<15:18,  9.00it/s, est. speed input: 3293.06 toks/s, output: 784.12 toks/s]
Processed prompts:  17%|█▋        | 1739/10000 [02:43<13:30, 10.19it/s, est. speed input: 3301.15 toks/s, output: 784.78 toks/s]
Processed prompts:  17%|█▋        | 1741/10000 [02:43<16:13,  8.48it/s, est. speed input: 3304.61 toks/s, output: 784.15 toks/s]
Processed prompts:  17%|█▋        | 1745/10000 [02:44<18:45,  7.33it/s, est. speed input: 3302.97 toks/s, output: 783.02 toks/s]
Processed prompts:  18%|█▊        | 1750/10000 [02:44<15:24,  8.92it/s, est. speed input: 3321.91 toks/s, output: 784.13 toks/s]
Processed prompts:  18%|█▊        | 1753/10000 [02:45<15:21,  8.95it/s, est. speed input: 3322.29 toks/s, output: 783.99 toks/s]
Processed prompts:  18%|█▊        | 1756/10000 [02:45<15:17,  8.99it/s, est. speed input: 3328.34 toks/s, output: 783.90 toks/s]
Processed prompts:  18%|█▊        | 1762/10000 [02:45<12:00, 11.43it/s, est. speed input: 3339.90 toks/s, output: 785.47 toks/s]
Processed prompts:  18%|█▊        | 1764/10000 [02:46<17:56,  7.65it/s, est. speed input: 3333.95 toks/s, output: 783.52 toks/s]
Processed prompts:  18%|█▊        | 1766/10000 [02:46<17:21,  7.90it/s, est. speed input: 3334.40 toks/s, output: 783.46 toks/s]
Processed prompts:  18%|█▊        | 1767/10000 [02:46<19:48,  6.93it/s, est. speed input: 3330.62 toks/s, output: 782.72 toks/s]
Processed prompts:  18%|█▊        | 1768/10000 [02:47<21:22,  6.42it/s, est. speed input: 3329.87 toks/s, output: 782.63 toks/s]
Processed prompts:  18%|█▊        | 1770/10000 [02:47<19:13,  7.14it/s, est. speed input: 3328.98 toks/s, output: 782.65 toks/s]
Processed prompts:  18%|█▊        | 1772/10000 [02:47<19:18,  7.11it/s, est. speed input: 3327.29 toks/s, output: 782.22 toks/s]
Processed prompts:  18%|█▊        | 1774/10000 [02:47<18:37,  7.36it/s, est. speed input: 3327.33 toks/s, output: 781.97 toks/s]
Processed prompts:  18%|█▊        | 1776/10000 [02:48<18:33,  7.39it/s, est. speed input: 3326.78 toks/s, output: 781.66 toks/s]
Processed prompts:  18%|█▊        | 1777/10000 [02:48<21:45,  6.30it/s, est. speed input: 3323.81 toks/s, output: 780.89 toks/s]
Processed prompts:  18%|█▊        | 1779/10000 [02:48<19:28,  7.04it/s, est. speed input: 3324.15 toks/s, output: 780.82 toks/s]
Processed prompts:  18%|█▊        | 1784/10000 [02:48<12:53, 10.63it/s, est. speed input: 3329.33 toks/s, output: 781.79 toks/s]
Processed prompts:  18%|█▊        | 1788/10000 [02:49<14:06,  9.70it/s, est. speed input: 3329.65 toks/s, output: 781.45 toks/s]
Processed prompts:  18%|█▊        | 1793/10000 [02:49<13:00, 10.51it/s, est. speed input: 3333.77 toks/s, output: 782.08 toks/s]
Processed prompts:  18%|█▊        | 1795/10000 [02:50<16:02,  8.52it/s, est. speed input: 3329.28 toks/s, output: 781.18 toks/s]
Processed prompts:  18%|█▊        | 1798/10000 [02:50<14:45,  9.27it/s, est. speed input: 3331.17 toks/s, output: 781.32 toks/s]
Processed prompts:  18%|█▊        | 1801/10000 [02:50<14:51,  9.20it/s, est. speed input: 3332.04 toks/s, output: 781.23 toks/s]
Processed prompts:  18%|█▊        | 1802/10000 [02:51<18:41,  7.31it/s, est. speed input: 3327.04 toks/s, output: 780.16 toks/s]
Processed prompts:  18%|█▊        | 1804/10000 [02:51<17:19,  7.88it/s, est. speed input: 3327.90 toks/s, output: 780.12 toks/s]
Processed prompts:  18%|█▊        | 1806/10000 [02:51<17:40,  7.73it/s, est. speed input: 3326.44 toks/s, output: 779.81 toks/s]
Processed prompts:  18%|█▊        | 1808/10000 [02:51<17:54,  7.62it/s, est. speed input: 3326.07 toks/s, output: 779.39 toks/s]
Processed prompts:  18%|█▊        | 1809/10000 [02:52<20:55,  6.52it/s, est. speed input: 3323.19 toks/s, output: 778.84 toks/s]
Processed prompts:  18%|█▊        | 1810/10000 [02:52<22:29,  6.07it/s, est. speed input: 3321.34 toks/s, output: 778.34 toks/s]
Processed prompts:  18%|█▊        | 1813/10000 [02:52<16:38,  8.20it/s, est. speed input: 3324.55 toks/s, output: 778.83 toks/s]
Processed prompts:  18%|█▊        | 1816/10000 [02:52<16:02,  8.50it/s, est. speed input: 3324.18 toks/s, output: 778.89 toks/s]
Processed prompts:  18%|█▊        | 1820/10000 [02:53<14:14,  9.58it/s, est. speed input: 3325.35 toks/s, output: 779.33 toks/s]
Processed prompts:  18%|█▊        | 1825/10000 [02:53<12:41, 10.73it/s, est. speed input: 3329.43 toks/s, output: 779.78 toks/s]
Processed prompts:  18%|█▊        | 1827/10000 [02:54<16:19,  8.34it/s, est. speed input: 3325.15 toks/s, output: 778.73 toks/s]
Processed prompts:  18%|█▊        | 1830/10000 [02:54<14:05,  9.66it/s, est. speed input: 3328.45 toks/s, output: 779.12 toks/s]
Processed prompts:  18%|█▊        | 1832/10000 [02:54<17:12,  7.91it/s, est. speed input: 3323.65 toks/s, output: 778.26 toks/s]
Processed prompts:  18%|█▊        | 1841/10000 [02:54<08:59, 15.11it/s, est. speed input: 3339.73 toks/s, output: 781.26 toks/s]
Processed prompts:  18%|█▊        | 1843/10000 [02:55<15:49,  8.59it/s, est. speed input: 3329.53 toks/s, output: 778.73 toks/s]
Processed prompts:  18%|█▊        | 1845/10000 [02:55<15:33,  8.73it/s, est. speed input: 3329.96 toks/s, output: 778.67 toks/s]
Processed prompts:  18%|█▊        | 1848/10000 [02:56<14:32,  9.35it/s, est. speed input: 3331.75 toks/s, output: 778.88 toks/s]
Processed prompts:  19%|█▊        | 1852/10000 [02:56<14:19,  9.48it/s, est. speed input: 3331.83 toks/s, output: 779.01 toks/s]
Processed prompts:  19%|█▊        | 1856/10000 [02:57<13:51,  9.79it/s, est. speed input: 3332.25 toks/s, output: 779.20 toks/s]
Processed prompts:  19%|█▊        | 1861/10000 [02:57<12:51, 10.55it/s, est. speed input: 3334.38 toks/s, output: 779.61 toks/s]
Processed prompts:  19%|█▊        | 1863/10000 [02:57<16:05,  8.42it/s, est. speed input: 3330.09 toks/s, output: 778.52 toks/s]
Processed prompts:  19%|█▊        | 1866/10000 [02:58<14:57,  9.06it/s, est. speed input: 3330.39 toks/s, output: 778.87 toks/s]
Processed prompts:  19%|█▊        | 1871/10000 [02:58<12:36, 10.75it/s, est. speed input: 3334.78 toks/s, output: 779.65 toks/s]
Processed prompts:  19%|█▊        | 1874/10000 [02:58<14:42,  9.21it/s, est. speed input: 3332.83 toks/s, output: 779.00 toks/s]
Processed prompts:  19%|█▉        | 1878/10000 [02:59<13:36,  9.95it/s, est. speed input: 3335.82 toks/s, output: 779.30 toks/s]
Processed prompts:  19%|█▉        | 1883/10000 [02:59<12:31, 10.80it/s, est. speed input: 3339.49 toks/s, output: 779.78 toks/s]
Processed prompts:  19%|█▉        | 1886/10000 [03:00<14:33,  9.29it/s, est. speed input: 3336.14 toks/s, output: 779.12 toks/s]
Processed prompts:  19%|█▉        | 1888/10000 [03:00<16:10,  8.36it/s, est. speed input: 3333.44 toks/s, output: 778.59 toks/s]
Processed prompts:  19%|█▉        | 1890/10000 [03:00<16:26,  8.22it/s, est. speed input: 3333.14 toks/s, output: 778.33 toks/s]
Processed prompts:  19%|█▉        | 1892/10000 [03:01<16:54,  7.99it/s, est. speed input: 3331.75 toks/s, output: 778.13 toks/s]
Processed prompts:  19%|█▉        | 1898/10000 [03:01<11:30, 11.74it/s, est. speed input: 3339.70 toks/s, output: 779.87 toks/s]
Processed prompts:  19%|█▉        | 1901/10000 [03:01<14:43,  9.17it/s, est. speed input: 3335.70 toks/s, output: 778.93 toks/s]
Processed prompts:  19%|█▉        | 1904/10000 [03:02<14:29,  9.32it/s, est. speed input: 3335.95 toks/s, output: 779.13 toks/s]
Processed prompts:  19%|█▉        | 1906/10000 [03:02<16:05,  8.39it/s, est. speed input: 3334.18 toks/s, output: 778.68 toks/s]
Processed prompts:  19%|█▉        | 1909/10000 [03:02<14:56,  9.02it/s, est. speed input: 3335.08 toks/s, output: 778.81 toks/s]
Processed prompts:  19%|█▉        | 1911/10000 [03:03<16:27,  8.19it/s, est. speed input: 3333.62 toks/s, output: 778.33 toks/s]
Processed prompts:  19%|█▉        | 1916/10000 [03:03<12:17, 10.96it/s, est. speed input: 3339.51 toks/s, output: 779.45 toks/s]
Processed prompts:  19%|█▉        | 1918/10000 [03:03<15:45,  8.54it/s, est. speed input: 3335.37 toks/s, output: 778.52 toks/s]
Processed prompts:  19%|█▉        | 1921/10000 [03:04<14:34,  9.24it/s, est. speed input: 3336.92 toks/s, output: 778.73 toks/s]
Processed prompts:  19%|█▉        | 1925/10000 [03:04<13:22, 10.06it/s, est. speed input: 3339.73 toks/s, output: 779.08 toks/s]
Processed prompts:  19%|█▉        | 1930/10000 [03:04<12:23, 10.85it/s, est. speed input: 3343.58 toks/s, output: 779.43 toks/s]
Processed prompts:  19%|█▉        | 1935/10000 [03:05<12:20, 10.89it/s, est. speed input: 3346.62 toks/s, output: 779.68 toks/s]
Processed prompts:  19%|█▉        | 1938/10000 [03:05<14:16,  9.41it/s, est. speed input: 3344.37 toks/s, output: 779.08 toks/s]
Processed prompts:  19%|█▉        | 1941/10000 [03:06<14:30,  9.26it/s, est. speed input: 3344.46 toks/s, output: 779.09 toks/s]
Processed prompts:  19%|█▉        | 1943/10000 [03:06<15:38,  8.59it/s, est. speed input: 3343.15 toks/s, output: 779.12 toks/s]
Processed prompts:  19%|█▉        | 1947/10000 [03:06<17:35,  7.63it/s, est. speed input: 3339.06 toks/s, output: 778.29 toks/s]
Processed prompts:  19%|█▉        | 1949/10000 [03:07<15:23,  8.72it/s, est. speed input: 3341.75 toks/s, output: 778.92 toks/s]
Processed prompts:  20%|█▉        | 1955/10000 [03:07<11:03, 12.13it/s, est. speed input: 3347.39 toks/s, output: 780.60 toks/s]
Processed prompts:  20%|█▉        | 1957/10000 [03:08<18:37,  7.20it/s, est. speed input: 3337.62 toks/s, output: 778.26 toks/s]
Processed prompts:  20%|█▉        | 1964/10000 [03:08<14:15,  9.39it/s, est. speed input: 3343.48 toks/s, output: 779.37 toks/s]
Processed prompts:  20%|█▉        | 1968/10000 [03:08<13:21, 10.02it/s, est. speed input: 3344.97 toks/s, output: 779.59 toks/s]
Processed prompts:  20%|█▉        | 1970/10000 [03:09<15:22,  8.71it/s, est. speed input: 3342.03 toks/s, output: 779.00 toks/s]
Processed prompts:  20%|█▉        | 1972/10000 [03:09<15:51,  8.44it/s, est. speed input: 3340.94 toks/s, output: 778.83 toks/s]
Processed prompts:  20%|█▉        | 1975/10000 [03:09<14:54,  8.97it/s, est. speed input: 3342.07 toks/s, output: 778.89 toks/s]
Processed prompts:  20%|█▉        | 1978/10000 [03:10<14:52,  8.99it/s, est. speed input: 3342.48 toks/s, output: 779.07 toks/s]
Processed prompts:  20%|█▉        | 1980/10000 [03:10<16:24,  8.15it/s, est. speed input: 3339.60 toks/s, output: 778.72 toks/s]
Processed prompts:  20%|█▉        | 1981/10000 [03:10<19:00,  7.03it/s, est. speed input: 3336.97 toks/s, output: 778.27 toks/s]
Processed prompts:  20%|█▉        | 1984/10000 [03:11<15:34,  8.58it/s, est. speed input: 3338.21 toks/s, output: 778.81 toks/s]
Processed prompts:  20%|█▉        | 1986/10000 [03:11<17:29,  7.64it/s, est. speed input: 3336.36 toks/s, output: 778.60 toks/s]
Processed prompts:  20%|█▉        | 1989/10000 [03:11<15:35,  8.56it/s, est. speed input: 3337.35 toks/s, output: 778.60 toks/s]
Processed prompts:  20%|█▉        | 1991/10000 [03:11<16:55,  7.89it/s, est. speed input: 3336.26 toks/s, output: 778.28 toks/s]
Processed prompts:  20%|█▉        | 1994/10000 [03:12<20:37,  6.47it/s, est. speed input: 3331.61 toks/s, output: 777.12 toks/s]
Processed prompts:  20%|█▉        | 1997/10000 [03:12<19:08,  6.97it/s, est. speed input: 3331.23 toks/s, output: 777.07 toks/s]
Processed prompts:  20%|██        | 2003/10000 [03:13<12:09, 10.96it/s, est. speed input: 3340.63 toks/s, output: 778.90 toks/s]
Processed prompts:  20%|██        | 2006/10000 [03:13<14:55,  8.93it/s, est. speed input: 3338.18 toks/s, output: 777.93 toks/s]
Processed prompts:  20%|██        | 2008/10000 [03:14<16:20,  8.15it/s, est. speed input: 3336.63 toks/s, output: 777.70 toks/s]
Processed prompts:  20%|██        | 2012/10000 [03:14<13:11, 10.09it/s, est. speed input: 3341.04 toks/s, output: 778.53 toks/s]
Processed prompts:  20%|██        | 2014/10000 [03:14<16:24,  8.11it/s, est. speed input: 3337.71 toks/s, output: 777.81 toks/s]
Processed prompts:  20%|██        | 2016/10000 [03:14<16:32,  8.05it/s, est. speed input: 3337.39 toks/s, output: 777.75 toks/s]
Processed prompts:  20%|██        | 2019/10000 [03:15<15:03,  8.83it/s, est. speed input: 3338.73 toks/s, output: 777.96 toks/s]
Processed prompts:  20%|██        | 2025/10000 [03:15<11:26, 11.62it/s, est. speed input: 3344.23 toks/s, output: 779.19 toks/s]
Processed prompts:  20%|██        | 2030/10000 [03:16<12:35, 10.55it/s, est. speed input: 3344.63 toks/s, output: 779.44 toks/s]
Processed prompts:  20%|██        | 2033/10000 [03:16<14:30,  9.15it/s, est. speed input: 3342.60 toks/s, output: 779.10 toks/s]
Processed prompts:  20%|██        | 2037/10000 [03:16<13:18,  9.98it/s, est. speed input: 3344.70 toks/s, output: 779.67 toks/s]
Processed prompts:  20%|██        | 2039/10000 [03:17<15:45,  8.42it/s, est. speed input: 3341.55 toks/s, output: 779.13 toks/s]
Processed prompts:  20%|██        | 2042/10000 [03:17<18:48,  7.05it/s, est. speed input: 3337.42 toks/s, output: 778.09 toks/s]
Processed prompts:  20%|██        | 2044/10000 [03:17<16:14,  8.17it/s, est. speed input: 3340.16 toks/s, output: 778.52 toks/s]
Processed prompts:  20%|██        | 2046/10000 [03:18<16:43,  7.92it/s, est. speed input: 3339.42 toks/s, output: 778.28 toks/s]
Processed prompts:  20%|██        | 2050/10000 [03:18<13:36,  9.74it/s, est. speed input: 3343.10 toks/s, output: 778.98 toks/s]
Processed prompts:  21%|██        | 2053/10000 [03:18<15:17,  8.66it/s, est. speed input: 3342.45 toks/s, output: 778.53 toks/s]
Processed prompts:  21%|██        | 2056/10000 [03:19<15:03,  8.79it/s, est. speed input: 3342.73 toks/s, output: 778.48 toks/s]
Processed prompts:  21%|██        | 2058/10000 [03:19<16:33,  8.00it/s, est. speed input: 3341.10 toks/s, output: 777.97 toks/s]
Processed prompts:  21%|██        | 2059/10000 [03:19<19:12,  6.89it/s, est. speed input: 3338.66 toks/s, output: 777.40 toks/s]
Processed prompts:  21%|██        | 2066/10000 [03:20<10:28, 12.62it/s, est. speed input: 3350.15 toks/s, output: 779.54 toks/s]
Processed prompts:  21%|██        | 2069/10000 [03:20<14:15,  9.27it/s, est. speed input: 3346.71 toks/s, output: 778.57 toks/s]
Processed prompts:  21%|██        | 2072/10000 [03:20<14:06,  9.36it/s, est. speed input: 3347.98 toks/s, output: 778.84 toks/s]
Processed prompts:  21%|██        | 2078/10000 [03:21<11:06, 11.88it/s, est. speed input: 3355.15 toks/s, output: 780.33 toks/s]
Processed prompts:  21%|██        | 2081/10000 [03:21<14:12,  9.29it/s, est. speed input: 3352.00 toks/s, output: 779.42 toks/s]
Processed prompts:  21%|██        | 2085/10000 [03:22<13:07, 10.05it/s, est. speed input: 3354.77 toks/s, output: 779.65 toks/s]
Processed prompts:  21%|██        | 2087/10000 [03:22<15:22,  8.58it/s, est. speed input: 3352.53 toks/s, output: 779.03 toks/s]
Processed prompts:  21%|██        | 2089/10000 [03:22<15:46,  8.36it/s, est. speed input: 3352.08 toks/s, output: 778.78 toks/s]
Processed prompts:  21%|██        | 2090/10000 [03:23<18:18,  7.20it/s, est. speed input: 3349.69 toks/s, output: 778.25 toks/s]
Processed prompts:  21%|██        | 2096/10000 [03:23<11:11, 11.76it/s, est. speed input: 3358.12 toks/s, output: 779.88 toks/s]
Processed prompts:  21%|██        | 2101/10000 [03:23<12:17, 10.71it/s, est. speed input: 3358.92 toks/s, output: 779.68 toks/s]
Processed prompts:  21%|██        | 2103/10000 [03:24<15:36,  8.43it/s, est. speed input: 3355.07 toks/s, output: 778.77 toks/s]
Processed prompts:  21%|██        | 2109/10000 [03:24<11:21, 11.58it/s, est. speed input: 3362.17 toks/s, output: 780.23 toks/s]
Processed prompts:  21%|██        | 2115/10000 [03:25<10:58, 11.97it/s, est. speed input: 3366.69 toks/s, output: 780.75 toks/s]
Processed prompts:  21%|██        | 2117/10000 [03:25<13:34,  9.68it/s, est. speed input: 3363.40 toks/s, output: 780.10 toks/s]
Processed prompts:  21%|██        | 2122/10000 [03:25<11:06, 11.82it/s, est. speed input: 3369.73 toks/s, output: 781.19 toks/s]
Processed prompts:  21%|██▏       | 2125/10000 [03:26<11:52, 11.05it/s, est. speed input: 3370.35 toks/s, output: 781.05 toks/s]
Processed prompts:  21%|██▏       | 2130/10000 [03:26<10:37, 12.34it/s, est. speed input: 3374.33 toks/s, output: 781.73 toks/s]
Processed prompts:  21%|██▏       | 2137/10000 [03:26<09:18, 14.08it/s, est. speed input: 3381.58 toks/s, output: 782.99 toks/s]
Processed prompts:  21%|██▏       | 2140/10000 [03:27<11:00, 11.90it/s, est. speed input: 3380.74 toks/s, output: 782.75 toks/s]
Processed prompts:  21%|██▏       | 2146/10000 [03:27<09:01, 14.50it/s, est. speed input: 3387.81 toks/s, output: 784.23 toks/s]
Processed prompts:  21%|██▏       | 2148/10000 [03:28<13:23,  9.77it/s, est. speed input: 3382.02 toks/s, output: 782.82 toks/s]
Processed prompts:  22%|██▏       | 2150/10000 [03:28<12:13, 10.70it/s, est. speed input: 3384.55 toks/s, output: 783.25 toks/s]
Processed prompts:  22%|██▏       | 2153/10000 [03:28<15:52,  8.24it/s, est. speed input: 3380.81 toks/s, output: 782.20 toks/s]
Processed prompts:  22%|██▏       | 2157/10000 [03:29<14:13,  9.19it/s, est. speed input: 3383.04 toks/s, output: 782.29 toks/s]
Processed prompts:  22%|██▏       | 2159/10000 [03:29<14:36,  8.95it/s, est. speed input: 3383.40 toks/s, output: 782.30 toks/s]
Processed prompts:  22%|██▏       | 2161/10000 [03:29<14:15,  9.16it/s, est. speed input: 3384.17 toks/s, output: 782.39 toks/s]
Processed prompts:  22%|██▏       | 2163/10000 [03:29<14:52,  8.78it/s, est. speed input: 3384.02 toks/s, output: 782.55 toks/s]
Processed prompts:  22%|██▏       | 2166/10000 [03:30<13:38,  9.57it/s, est. speed input: 3386.21 toks/s, output: 782.81 toks/s]
Processed prompts:  22%|██▏       | 2168/10000 [03:30<14:50,  8.79it/s, est. speed input: 3385.99 toks/s, output: 782.78 toks/s]
Processed prompts:  22%|██▏       | 2171/10000 [03:30<12:24, 10.52it/s, est. speed input: 3388.69 toks/s, output: 783.33 toks/s]
Processed prompts:  22%|██▏       | 2174/10000 [03:30<12:52, 10.13it/s, est. speed input: 3389.04 toks/s, output: 783.30 toks/s]
Processed prompts:  22%|██▏       | 2178/10000 [03:31<11:19, 11.51it/s, est. speed input: 3392.77 toks/s, output: 783.87 toks/s]
Processed prompts:  22%|██▏       | 2180/10000 [03:31<15:47,  8.26it/s, est. speed input: 3388.84 toks/s, output: 782.84 toks/s]
Processed prompts:  22%|██▏       | 2184/10000 [03:31<14:06,  9.23it/s, est. speed input: 3391.50 toks/s, output: 783.34 toks/s]
Processed prompts:  22%|██▏       | 2188/10000 [03:32<12:16, 10.61it/s, est. speed input: 3394.98 toks/s, output: 783.90 toks/s]
Processed prompts:  22%|██▏       | 2194/10000 [03:32<09:52, 13.17it/s, est. speed input: 3401.78 toks/s, output: 784.99 toks/s]
Processed prompts:  22%|██▏       | 2196/10000 [03:33<14:59,  8.68it/s, est. speed input: 3396.07 toks/s, output: 783.73 toks/s]
Processed prompts:  22%|██▏       | 2201/10000 [03:33<12:58, 10.02it/s, est. speed input: 3399.66 toks/s, output: 784.24 toks/s]
Processed prompts:  22%|██▏       | 2205/10000 [03:33<11:38, 11.16it/s, est. speed input: 3402.11 toks/s, output: 784.73 toks/s]
Processed prompts:  22%|██▏       | 2209/10000 [03:34<11:11, 11.60it/s, est. speed input: 3403.63 toks/s, output: 784.84 toks/s]
Processed prompts:  22%|██▏       | 2211/10000 [03:34<13:08,  9.88it/s, est. speed input: 3402.29 toks/s, output: 784.64 toks/s]
Processed prompts:  22%|██▏       | 2213/10000 [03:34<12:39, 10.25it/s, est. speed input: 3403.65 toks/s, output: 785.25 toks/s]
Processed prompts:  22%|██▏       | 2215/10000 [03:35<17:35,  7.38it/s, est. speed input: 3398.79 toks/s, output: 784.06 toks/s]
Processed prompts:  22%|██▏       | 2218/10000 [03:35<13:19,  9.73it/s, est. speed input: 3403.13 toks/s, output: 784.92 toks/s]
Processed prompts:  22%|██▏       | 2220/10000 [03:35<14:35,  8.89it/s, est. speed input: 3402.84 toks/s, output: 784.70 toks/s]
Processed prompts:  22%|██▏       | 2222/10000 [03:35<14:45,  8.78it/s, est. speed input: 3403.35 toks/s, output: 784.64 toks/s]
Processed prompts:  22%|██▏       | 2224/10000 [03:36<15:27,  8.38it/s, est. speed input: 3403.37 toks/s, output: 784.53 toks/s]
Processed prompts:  22%|██▏       | 2226/10000 [03:36<14:08,  9.16it/s, est. speed input: 3404.83 toks/s, output: 784.82 toks/s]
Processed prompts:  22%|██▏       | 2231/10000 [03:36<10:21, 12.51it/s, est. speed input: 3411.11 toks/s, output: 785.87 toks/s]
Processed prompts:  22%|██▏       | 2235/10000 [03:36<10:58, 11.78it/s, est. speed input: 3412.25 toks/s, output: 786.14 toks/s]
Processed prompts:  22%|██▏       | 2239/10000 [03:37<10:42, 12.07it/s, est. speed input: 3414.90 toks/s, output: 786.67 toks/s]
Processed prompts:  22%|██▏       | 2242/10000 [03:37<10:58, 11.78it/s, est. speed input: 3415.95 toks/s, output: 786.86 toks/s]
Processed prompts:  22%|██▏       | 2247/10000 [03:37<10:01, 12.89it/s, est. speed input: 3419.55 toks/s, output: 787.55 toks/s]
Processed prompts:  22%|██▎       | 2250/10000 [03:37<08:52, 14.54it/s, est. speed input: 3423.20 toks/s, output: 788.26 toks/s]
Processed prompts:  23%|██▎       | 2253/10000 [03:38<09:34, 13.48it/s, est. speed input: 3425.10 toks/s, output: 788.63 toks/s]
Processed prompts:  23%|██▎       | 2255/10000 [03:38<11:12, 11.52it/s, est. speed input: 3424.81 toks/s, output: 788.49 toks/s]
Processed prompts:  23%|██▎       | 2260/10000 [03:38<08:45, 14.72it/s, est. speed input: 3431.22 toks/s, output: 789.49 toks/s]
Processed prompts:  23%|██▎       | 2263/10000 [03:38<10:46, 11.97it/s, est. speed input: 3430.71 toks/s, output: 789.37 toks/s]
Processed prompts:  23%|██▎       | 2266/10000 [03:39<10:57, 11.77it/s, est. speed input: 3431.59 toks/s, output: 789.39 toks/s]
Processed prompts:  23%|██▎       | 2268/10000 [03:39<12:16, 10.49it/s, est. speed input: 3430.35 toks/s, output: 789.18 toks/s]
Processed prompts:  23%|██▎       | 2271/10000 [03:39<11:48, 10.91it/s, est. speed input: 3432.68 toks/s, output: 789.43 toks/s]
Processed prompts:  23%|██▎       | 2273/10000 [03:40<12:52, 10.00it/s, est. speed input: 3432.64 toks/s, output: 789.54 toks/s]
Processed prompts:  23%|██▎       | 2278/10000 [03:40<09:26, 13.63it/s, est. speed input: 3438.45 toks/s, output: 790.76 toks/s]
Processed prompts:  23%|██▎       | 2282/10000 [03:40<10:27, 12.29it/s, est. speed input: 3440.17 toks/s, output: 791.07 toks/s]
Processed prompts:  23%|██▎       | 2284/10000 [03:40<12:41, 10.14it/s, est. speed input: 3438.82 toks/s, output: 790.53 toks/s]
Processed prompts:  23%|██▎       | 2286/10000 [03:41<13:16,  9.68it/s, est. speed input: 3439.21 toks/s, output: 790.51 toks/s]
Processed prompts:  23%|██▎       | 2288/10000 [03:41<13:10,  9.75it/s, est. speed input: 3439.09 toks/s, output: 790.65 toks/s]
Processed prompts:  23%|██▎       | 2292/10000 [03:41<11:17, 11.38it/s, est. speed input: 3441.84 toks/s, output: 791.23 toks/s]
Processed prompts:  23%|██▎       | 2294/10000 [03:42<14:08,  9.08it/s, est. speed input: 3439.35 toks/s, output: 790.64 toks/s]
Processed prompts:  23%|██▎       | 2296/10000 [03:42<13:46,  9.33it/s, est. speed input: 3439.68 toks/s, output: 790.77 toks/s]
Processed prompts:  23%|██▎       | 2302/10000 [03:42<09:45, 13.14it/s, est. speed input: 3445.77 toks/s, output: 792.00 toks/s]
Processed prompts:  23%|██▎       | 2305/10000 [03:42<10:58, 11.68it/s, est. speed input: 3445.42 toks/s, output: 791.90 toks/s]
Processed prompts:  23%|██▎       | 2310/10000 [03:43<09:34, 13.38it/s, est. speed input: 3449.67 toks/s, output: 792.74 toks/s]
Processed prompts:  23%|██▎       | 2313/10000 [03:43<11:18, 11.33it/s, est. speed input: 3449.63 toks/s, output: 792.69 toks/s]
Processed prompts:  23%|██▎       | 2315/10000 [03:43<12:28, 10.26it/s, est. speed input: 3449.34 toks/s, output: 792.53 toks/s]
Processed prompts:  23%|██▎       | 2318/10000 [03:43<11:15, 11.37it/s, est. speed input: 3451.94 toks/s, output: 793.00 toks/s]
Processed prompts:  23%|██▎       | 2320/10000 [03:44<13:30,  9.47it/s, est. speed input: 3450.47 toks/s, output: 792.74 toks/s]
Processed prompts:  23%|██▎       | 2324/10000 [03:44<11:21, 11.26it/s, est. speed input: 3453.31 toks/s, output: 793.34 toks/s]
Processed prompts:  23%|██▎       | 2326/10000 [03:45<15:22,  8.32it/s, est. speed input: 3449.21 toks/s, output: 792.37 toks/s]
Processed prompts:  23%|██▎       | 2328/10000 [03:45<14:58,  8.54it/s, est. speed input: 3449.99 toks/s, output: 792.41 toks/s]
Processed prompts:  23%|██▎       | 2332/10000 [03:45<16:29,  7.75it/s, est. speed input: 3447.81 toks/s, output: 791.77 toks/s]
Processed prompts:  23%|██▎       | 2342/10000 [03:46<08:57, 14.24it/s, est. speed input: 3458.32 toks/s, output: 794.48 toks/s]
Processed prompts:  23%|██▎       | 2344/10000 [03:46<12:42, 10.04it/s, est. speed input: 3453.18 toks/s, output: 793.30 toks/s]
Processed prompts:  23%|██▎       | 2346/10000 [03:46<12:49,  9.94it/s, est. speed input: 3453.27 toks/s, output: 793.37 toks/s]
Processed prompts:  24%|██▎       | 2350/10000 [03:47<10:54, 11.69it/s, est. speed input: 3456.04 toks/s, output: 794.05 toks/s]
Processed prompts:  24%|██▎       | 2354/10000 [03:47<10:49, 11.77it/s, est. speed input: 3456.48 toks/s, output: 794.31 toks/s]
Processed prompts:  24%|██▎       | 2358/10000 [03:47<10:31, 12.10it/s, est. speed input: 3457.49 toks/s, output: 794.71 toks/s]
Processed prompts:  24%|██▎       | 2361/10000 [03:48<10:57, 11.61it/s, est. speed input: 3458.94 toks/s, output: 795.11 toks/s]
Processed prompts:  24%|██▎       | 2366/10000 [03:48<14:00,  9.08it/s, est. speed input: 3455.00 toks/s, output: 794.50 toks/s]
Processed prompts:  24%|██▎       | 2374/10000 [03:49<11:00, 11.55it/s, est. speed input: 3458.77 toks/s, output: 795.92 toks/s]
Processed prompts:  24%|██▍       | 2377/10000 [03:49<10:59, 11.56it/s, est. speed input: 3458.64 toks/s, output: 796.23 toks/s]
Processed prompts:  24%|██▍       | 2381/10000 [03:49<10:14, 12.40it/s, est. speed input: 3460.05 toks/s, output: 796.68 toks/s]
Processed prompts:  24%|██▍       | 2386/10000 [03:50<09:28, 13.38it/s, est. speed input: 3461.79 toks/s, output: 797.58 toks/s]
Processed prompts:  24%|██▍       | 2390/10000 [03:50<12:59,  9.76it/s, est. speed input: 3455.81 toks/s, output: 796.47 toks/s]
Processed prompts:  24%|██▍       | 2400/10000 [03:51<08:32, 14.84it/s, est. speed input: 3465.81 toks/s, output: 799.52 toks/s]
Processed prompts:  24%|██▍       | 2402/10000 [03:51<09:19, 13.57it/s, est. speed input: 3464.22 toks/s, output: 799.44 toks/s]
Processed prompts:  24%|██▍       | 2404/10000 [03:51<10:01, 12.62it/s, est. speed input: 3462.67 toks/s, output: 799.41 toks/s]
Processed prompts:  24%|██▍       | 2407/10000 [03:51<10:42, 11.83it/s, est. speed input: 3461.45 toks/s, output: 799.23 toks/s]
Processed prompts:  24%|██▍       | 2409/10000 [03:52<11:46, 10.75it/s, est. speed input: 3461.14 toks/s, output: 799.21 toks/s]
Processed prompts:  24%|██▍       | 2413/10000 [03:52<09:59, 12.66it/s, est. speed input: 3462.25 toks/s, output: 799.79 toks/s]
Processed prompts:  24%|██▍       | 2418/10000 [03:52<09:53, 12.78it/s, est. speed input: 3462.36 toks/s, output: 800.09 toks/s]
Processed prompts:  24%|██▍       | 2421/10000 [03:53<10:58, 11.51it/s, est. speed input: 3460.76 toks/s, output: 799.92 toks/s]
Processed prompts:  24%|██▍       | 2425/10000 [03:53<10:00, 12.62it/s, est. speed input: 3461.34 toks/s, output: 800.27 toks/s]
Processed prompts:  24%|██▍       | 2427/10000 [03:54<16:06,  7.84it/s, est. speed input: 3454.23 toks/s, output: 798.73 toks/s]
Processed prompts:  24%|██▍       | 2433/10000 [03:54<12:17, 10.25it/s, est. speed input: 3456.51 toks/s, output: 799.93 toks/s]
Processed prompts:  24%|██▍       | 2436/10000 [03:54<12:06, 10.42it/s, est. speed input: 3456.88 toks/s, output: 800.12 toks/s]
Processed prompts:  24%|██▍       | 2438/10000 [03:54<13:15,  9.50it/s, est. speed input: 3455.09 toks/s, output: 799.86 toks/s]
Processed prompts:  24%|██▍       | 2442/10000 [03:55<11:20, 11.11it/s, est. speed input: 3456.89 toks/s, output: 800.40 toks/s]
Processed prompts:  24%|██▍       | 2444/10000 [03:55<13:08,  9.58it/s, est. speed input: 3455.23 toks/s, output: 800.21 toks/s]
Processed prompts:  24%|██▍       | 2446/10000 [03:55<12:43,  9.90it/s, est. speed input: 3454.76 toks/s, output: 800.17 toks/s]
Processed prompts:  24%|██▍       | 2448/10000 [03:56<16:36,  7.58it/s, est. speed input: 3450.44 toks/s, output: 799.42 toks/s]
Processed prompts:  25%|██▍       | 2451/10000 [03:56<13:41,  9.19it/s, est. speed input: 3450.23 toks/s, output: 799.83 toks/s]
Processed prompts:  25%|██▍       | 2455/10000 [03:56<11:38, 10.81it/s, est. speed input: 3450.69 toks/s, output: 800.28 toks/s]
Processed prompts:  25%|██▍       | 2460/10000 [03:56<10:03, 12.50it/s, est. speed input: 3451.37 toks/s, output: 801.04 toks/s]
Processed prompts:  25%|██▍       | 2464/10000 [03:57<10:07, 12.40it/s, est. speed input: 3452.40 toks/s, output: 801.64 toks/s]
Processed prompts:  25%|██▍       | 2467/10000 [03:57<11:03, 11.35it/s, est. speed input: 3450.56 toks/s, output: 801.41 toks/s]
Processed prompts:  25%|██▍       | 2471/10000 [03:57<10:13, 12.28it/s, est. speed input: 3451.14 toks/s, output: 801.79 toks/s]
Processed prompts:  25%|██▍       | 2479/10000 [03:58<11:37, 10.79it/s, est. speed input: 3449.20 toks/s, output: 801.69 toks/s]
Processed prompts:  25%|██▍       | 2485/10000 [03:59<11:14, 11.14it/s, est. speed input: 3449.52 toks/s, output: 801.94 toks/s]
Processed prompts:  25%|██▍       | 2493/10000 [03:59<09:16, 13.49it/s, est. speed input: 3453.14 toks/s, output: 803.43 toks/s]
Processed prompts:  25%|██▍       | 2498/10000 [04:00<11:23, 10.97it/s, est. speed input: 3448.27 toks/s, output: 802.63 toks/s]
Processed prompts:  25%|██▌       | 2505/10000 [04:00<09:52, 12.65it/s, est. speed input: 3450.92 toks/s, output: 803.79 toks/s]
Processed prompts:  25%|██▌       | 2507/10000 [04:01<11:10, 11.17it/s, est. speed input: 3448.57 toks/s, output: 803.55 toks/s]
Processed prompts:  25%|██▌       | 2512/10000 [04:01<09:05, 13.72it/s, est. speed input: 3451.67 toks/s, output: 804.41 toks/s]
Processed prompts:  25%|██▌       | 2515/10000 [04:01<10:46, 11.57it/s, est. speed input: 3449.03 toks/s, output: 803.96 toks/s]
Processed prompts:  25%|██▌       | 2521/10000 [04:01<08:46, 14.19it/s, est. speed input: 3453.60 toks/s, output: 805.48 toks/s]
Processed prompts:  25%|██▌       | 2529/10000 [04:02<07:38, 16.29it/s, est. speed input: 3457.10 toks/s, output: 806.87 toks/s]
Processed prompts:  25%|██▌       | 2532/10000 [04:02<10:03, 12.36it/s, est. speed input: 3453.48 toks/s, output: 806.01 toks/s]
Processed prompts:  25%|██▌       | 2534/10000 [04:03<11:04, 11.23it/s, est. speed input: 3451.96 toks/s, output: 805.75 toks/s]
Processed prompts:  25%|██▌       | 2539/10000 [04:03<09:20, 13.32it/s, est. speed input: 3454.79 toks/s, output: 806.62 toks/s]
Processed prompts:  25%|██▌       | 2543/10000 [04:03<09:33, 12.99it/s, est. speed input: 3455.83 toks/s, output: 807.05 toks/s]
Processed prompts:  25%|██▌       | 2545/10000 [04:03<11:11, 11.10it/s, est. speed input: 3453.72 toks/s, output: 806.67 toks/s]
Processed prompts:  25%|██▌       | 2547/10000 [04:04<11:35, 10.71it/s, est. speed input: 3453.12 toks/s, output: 806.59 toks/s]
Processed prompts:  26%|██▌       | 2555/10000 [04:04<07:30, 16.52it/s, est. speed input: 3459.22 toks/s, output: 808.43 toks/s]
Processed prompts:  26%|██▌       | 2557/10000 [04:05<13:27,  9.22it/s, est. speed input: 3451.94 toks/s, output: 806.90 toks/s]
Processed prompts:  26%|██▌       | 2563/10000 [04:05<09:29, 13.07it/s, est. speed input: 3457.32 toks/s, output: 808.27 toks/s]
Processed prompts:  26%|██▌       | 2566/10000 [04:05<11:01, 11.24it/s, est. speed input: 3455.58 toks/s, output: 807.98 toks/s]
Processed prompts:  26%|██▌       | 2568/10000 [04:05<12:25,  9.97it/s, est. speed input: 3453.68 toks/s, output: 807.67 toks/s]
Processed prompts:  26%|██▌       | 2570/10000 [04:06<12:34,  9.85it/s, est. speed input: 3452.87 toks/s, output: 807.69 toks/s]
Processed prompts:  26%|██▌       | 2572/10000 [04:06<13:12,  9.38it/s, est. speed input: 3451.74 toks/s, output: 807.56 toks/s]
Processed prompts:  26%|██▌       | 2575/10000 [04:06<11:50, 10.45it/s, est. speed input: 3452.65 toks/s, output: 807.63 toks/s]
Processed prompts:  26%|██▌       | 2578/10000 [04:06<12:21, 10.01it/s, est. speed input: 3451.02 toks/s, output: 807.45 toks/s]
Processed prompts:  26%|██▌       | 2581/10000 [04:07<12:25,  9.95it/s, est. speed input: 3450.18 toks/s, output: 807.46 toks/s]
Processed prompts:  26%|██▌       | 2586/10000 [04:07<10:08, 12.17it/s, est. speed input: 3452.05 toks/s, output: 808.18 toks/s]
Processed prompts:  26%|██▌       | 2588/10000 [04:07<12:09, 10.16it/s, est. speed input: 3449.41 toks/s, output: 807.77 toks/s]
Processed prompts:  26%|██▌       | 2591/10000 [04:08<11:32, 10.69it/s, est. speed input: 3449.20 toks/s, output: 808.10 toks/s]
Processed prompts:  26%|██▌       | 2596/10000 [04:08<09:29, 13.01it/s, est. speed input: 3452.04 toks/s, output: 808.94 toks/s]
Processed prompts:  26%|██▌       | 2598/10000 [04:08<11:47, 10.46it/s, est. speed input: 3449.13 toks/s, output: 808.37 toks/s]
Processed prompts:  26%|██▌       | 2602/10000 [04:09<10:17, 11.99it/s, est. speed input: 3450.00 toks/s, output: 808.70 toks/s]
Processed prompts:  26%|██▌       | 2605/10000 [04:09<11:16, 10.94it/s, est. speed input: 3449.51 toks/s, output: 808.35 toks/s]
Processed prompts:  26%|██▌       | 2607/10000 [04:09<12:20,  9.99it/s, est. speed input: 3448.42 toks/s, output: 808.30 toks/s]
Processed prompts:  26%|██▌       | 2612/10000 [04:09<09:07, 13.50it/s, est. speed input: 3450.41 toks/s, output: 809.36 toks/s]
Processed prompts:  26%|██▌       | 2616/10000 [04:10<09:57, 12.36it/s, est. speed input: 3449.58 toks/s, output: 809.34 toks/s]
Processed prompts:  26%|██▌       | 2618/10000 [04:10<11:52, 10.36it/s, est. speed input: 3447.53 toks/s, output: 808.96 toks/s]
Processed prompts:  26%|██▌       | 2620/10000 [04:10<12:34,  9.78it/s, est. speed input: 3446.32 toks/s, output: 808.85 toks/s]
Processed prompts:  26%|██▌       | 2622/10000 [04:10<12:44,  9.66it/s, est. speed input: 3445.32 toks/s, output: 808.77 toks/s]
Processed prompts:  26%|██▌       | 2624/10000 [04:11<12:51,  9.57it/s, est. speed input: 3444.56 toks/s, output: 808.85 toks/s]
Processed prompts:  26%|██▋       | 2627/10000 [04:11<11:55, 10.30it/s, est. speed input: 3444.70 toks/s, output: 809.24 toks/s]
Processed prompts:  26%|██▋       | 2630/10000 [04:11<12:26,  9.88it/s, est. speed input: 3444.19 toks/s, output: 809.29 toks/s]
Processed prompts:  26%|██▋       | 2632/10000 [04:12<13:28,  9.11it/s, est. speed input: 3442.36 toks/s, output: 808.94 toks/s]
Processed prompts:  26%|██▋       | 2634/10000 [04:12<13:22,  9.18it/s, est. speed input: 3442.36 toks/s, output: 808.77 toks/s]
Processed prompts:  26%|██▋       | 2635/10000 [04:12<16:13,  7.57it/s, est. speed input: 3440.12 toks/s, output: 808.22 toks/s]
Processed prompts:  26%|██▋       | 2637/10000 [04:12<14:53,  8.24it/s, est. speed input: 3439.60 toks/s, output: 808.19 toks/s]
Processed prompts:  26%|██▋       | 2641/10000 [04:12<11:41, 10.49it/s, est. speed input: 3440.43 toks/s, output: 808.41 toks/s]
Processed prompts:  26%|██▋       | 2644/10000 [04:13<11:47, 10.40it/s, est. speed input: 3440.28 toks/s, output: 808.39 toks/s]
Processed prompts:  26%|██▋       | 2648/10000 [04:13<10:25, 11.76it/s, est. speed input: 3440.49 toks/s, output: 808.80 toks/s]
Processed prompts:  27%|██▋       | 2651/10000 [04:13<10:51, 11.27it/s, est. speed input: 3440.65 toks/s, output: 808.88 toks/s]
Processed prompts:  27%|██▋       | 2658/10000 [04:14<07:50, 15.60it/s, est. speed input: 3444.16 toks/s, output: 810.13 toks/s]
Processed prompts:  27%|██▋       | 2660/10000 [04:14<11:03, 11.06it/s, est. speed input: 3440.49 toks/s, output: 809.27 toks/s]
Processed prompts:  27%|██▋       | 2663/10000 [04:14<10:16, 11.90it/s, est. speed input: 3441.03 toks/s, output: 809.43 toks/s]
Processed prompts:  27%|██▋       | 2665/10000 [04:15<12:06, 10.10it/s, est. speed input: 3438.76 toks/s, output: 809.13 toks/s]
Processed prompts:  27%|██▋       | 2671/10000 [04:15<08:20, 14.64it/s, est. speed input: 3442.25 toks/s, output: 810.19 toks/s]
Processed prompts:  27%|██▋       | 2673/10000 [04:15<11:23, 10.71it/s, est. speed input: 3439.19 toks/s, output: 809.61 toks/s]
Processed prompts:  27%|██▋       | 2678/10000 [04:15<08:58, 13.60it/s, est. speed input: 3441.19 toks/s, output: 810.47 toks/s]
Processed prompts:  27%|██▋       | 2680/10000 [04:16<11:10, 10.91it/s, est. speed input: 3438.64 toks/s, output: 810.09 toks/s]
Processed prompts:  27%|██▋       | 2685/10000 [04:16<09:09, 13.31it/s, est. speed input: 3441.28 toks/s, output: 810.70 toks/s]
Processed prompts:  27%|██▋       | 2690/10000 [04:16<09:10, 13.29it/s, est. speed input: 3441.74 toks/s, output: 810.83 toks/s]
Processed prompts:  27%|██▋       | 2693/10000 [04:17<10:10, 11.97it/s, est. speed input: 3440.72 toks/s, output: 810.73 toks/s]
Processed prompts:  27%|██▋       | 2697/10000 [04:17<09:28, 12.84it/s, est. speed input: 3441.60 toks/s, output: 811.18 toks/s]
Processed prompts:  27%|██▋       | 2701/10000 [04:17<09:38, 12.62it/s, est. speed input: 3441.46 toks/s, output: 811.64 toks/s]
Processed prompts:  27%|██▋       | 2706/10000 [04:18<08:55, 13.63it/s, est. speed input: 3442.48 toks/s, output: 812.05 toks/s]
Processed prompts:  27%|██▋       | 2710/10000 [04:18<09:19, 13.03it/s, est. speed input: 3442.62 toks/s, output: 812.42 toks/s]
Processed prompts:  27%|██▋       | 2713/10000 [04:18<09:53, 12.27it/s, est. speed input: 3442.14 toks/s, output: 812.40 toks/s]
Processed prompts:  27%|██▋       | 2719/10000 [04:19<08:15, 14.70it/s, est. speed input: 3444.84 toks/s, output: 813.31 toks/s]
Processed prompts:  27%|██▋       | 2721/10000 [04:19<11:07, 10.90it/s, est. speed input: 3441.18 toks/s, output: 812.63 toks/s]
Processed prompts:  27%|██▋       | 2724/10000 [04:19<10:51, 11.16it/s, est. speed input: 3440.63 toks/s, output: 812.52 toks/s]
Processed prompts:  27%|██▋       | 2727/10000 [04:19<11:03, 10.97it/s, est. speed input: 3440.18 toks/s, output: 812.40 toks/s]
Processed prompts:  27%|██▋       | 2731/10000 [04:20<10:08, 11.95it/s, est. speed input: 3440.76 toks/s, output: 812.66 toks/s]
Processed prompts:  27%|██▋       | 2737/10000 [04:20<08:50, 13.70it/s, est. speed input: 3441.56 toks/s, output: 813.17 toks/s]
Processed prompts:  27%|██▋       | 2739/10000 [04:21<13:06,  9.23it/s, est. speed input: 3436.83 toks/s, output: 811.99 toks/s]
Processed prompts:  27%|██▋       | 2745/10000 [04:21<10:42, 11.29it/s, est. speed input: 3438.34 toks/s, output: 812.42 toks/s]
Processed prompts:  27%|██▋       | 2748/10000 [04:22<13:56,  8.67it/s, est. speed input: 3433.44 toks/s, output: 811.44 toks/s]
Processed prompts:  28%|██▊       | 2753/10000 [04:22<11:29, 10.51it/s, est. speed input: 3434.52 toks/s, output: 812.27 toks/s]
Processed prompts:  28%|██▊       | 2757/10000 [04:22<10:55, 11.04it/s, est. speed input: 3434.81 toks/s, output: 812.43 toks/s]
Processed prompts:  28%|██▊       | 2761/10000 [04:23<10:37, 11.35it/s, est. speed input: 3434.84 toks/s, output: 812.50 toks/s]
Processed prompts:  28%|██▊       | 2763/10000 [04:23<12:00, 10.05it/s, est. speed input: 3432.39 toks/s, output: 812.11 toks/s]
Processed prompts:  28%|██▊       | 2767/10000 [04:23<10:11, 11.84it/s, est. speed input: 3433.42 toks/s, output: 812.51 toks/s]
Processed prompts:  28%|██▊       | 2773/10000 [04:24<12:03,  9.99it/s, est. speed input: 3430.43 toks/s, output: 812.14 toks/s]
Processed prompts:  28%|██▊       | 2782/10000 [04:24<09:09, 13.15it/s, est. speed input: 3434.56 toks/s, output: 813.68 toks/s]
Processed prompts:  28%|██▊       | 2784/10000 [04:25<10:32, 11.40it/s, est. speed input: 3432.84 toks/s, output: 813.27 toks/s]
Processed prompts:  28%|██▊       | 2786/10000 [04:25<10:54, 11.02it/s, est. speed input: 3432.26 toks/s, output: 813.26 toks/s]
Processed prompts:  28%|██▊       | 2789/10000 [04:25<10:22, 11.59it/s, est. speed input: 3432.61 toks/s, output: 813.56 toks/s]
Processed prompts:  28%|██▊       | 2791/10000 [04:25<11:44, 10.23it/s, est. speed input: 3430.79 toks/s, output: 813.24 toks/s]
Processed prompts:  28%|██▊       | 2796/10000 [04:26<08:53, 13.51it/s, est. speed input: 3433.81 toks/s, output: 814.09 toks/s]
Processed prompts:  28%|██▊       | 2798/10000 [04:26<11:30, 10.44it/s, est. speed input: 3431.43 toks/s, output: 813.57 toks/s]
Processed prompts:  28%|██▊       | 2804/10000 [04:26<07:56, 15.12it/s, est. speed input: 3435.01 toks/s, output: 814.42 toks/s]
Processed prompts:  28%|██▊       | 2809/10000 [04:27<09:28, 12.66it/s, est. speed input: 3433.61 toks/s, output: 814.21 toks/s]
Processed prompts:  28%|██▊       | 2811/10000 [04:27<11:09, 10.74it/s, est. speed input: 3431.39 toks/s, output: 813.76 toks/s]
Processed prompts:  28%|██▊       | 2813/10000 [04:27<13:54,  8.62it/s, est. speed input: 3427.84 toks/s, output: 813.09 toks/s]
Processed prompts:  28%|██▊       | 2817/10000 [04:28<11:02, 10.84it/s, est. speed input: 3429.23 toks/s, output: 813.67 toks/s]
Processed prompts:  28%|██▊       | 2819/10000 [04:28<12:59,  9.22it/s, est. speed input: 3426.94 toks/s, output: 813.20 toks/s]
Processed prompts:  28%|██▊       | 2821/10000 [04:28<13:35,  8.81it/s, est. speed input: 3425.66 toks/s, output: 813.11 toks/s]
Processed prompts:  28%|██▊       | 2824/10000 [04:28<12:34,  9.51it/s, est. speed input: 3425.68 toks/s, output: 813.45 toks/s]
Processed prompts:  28%|██▊       | 2827/10000 [04:29<12:33,  9.52it/s, est. speed input: 3424.53 toks/s, output: 813.40 toks/s]
Processed prompts:  28%|██▊       | 2834/10000 [04:29<08:25, 14.19it/s, est. speed input: 3428.51 toks/s, output: 814.48 toks/s]
Processed prompts:  28%|██▊       | 2840/10000 [04:30<09:04, 13.15it/s, est. speed input: 3428.86 toks/s, output: 814.57 toks/s]
Processed prompts:  28%|██▊       | 2842/10000 [04:30<11:38, 10.25it/s, est. speed input: 3425.18 toks/s, output: 813.66 toks/s]
Processed prompts:  28%|██▊       | 2844/10000 [04:30<14:00,  8.51it/s, est. speed input: 3421.83 toks/s, output: 812.87 toks/s]
Processed prompts:  28%|██▊       | 2845/10000 [04:31<14:49,  8.04it/s, est. speed input: 3420.67 toks/s, output: 812.71 toks/s]
Processed prompts:  28%|██▊       | 2849/10000 [04:31<11:14, 10.60it/s, est. speed input: 3422.27 toks/s, output: 813.16 toks/s]
Processed prompts:  29%|██▊       | 2853/10000 [04:31<10:43, 11.11it/s, est. speed input: 3421.69 toks/s, output: 813.28 toks/s]
Processed prompts:  29%|██▊       | 2859/10000 [04:31<08:33, 13.90it/s, est. speed input: 3424.09 toks/s, output: 814.01 toks/s]
Processed prompts:  29%|██▊       | 2862/10000 [04:32<10:14, 11.62it/s, est. speed input: 3422.63 toks/s, output: 813.85 toks/s]
Processed prompts:  29%|██▊       | 2866/10000 [04:32<09:30, 12.51it/s, est. speed input: 3424.41 toks/s, output: 814.07 toks/s]
Processed prompts:  29%|██▊       | 2870/10000 [04:32<09:41, 12.27it/s, est. speed input: 3423.93 toks/s, output: 814.18 toks/s]
Processed prompts:  29%|██▉       | 2875/10000 [04:33<09:01, 13.17it/s, est. speed input: 3425.49 toks/s, output: 814.76 toks/s]
Processed prompts:  29%|██▉       | 2877/10000 [04:33<11:20, 10.47it/s, est. speed input: 3423.62 toks/s, output: 814.21 toks/s]
Processed prompts:  29%|██▉       | 2883/10000 [04:33<08:45, 13.54it/s, est. speed input: 3427.27 toks/s, output: 815.17 toks/s]
Processed prompts:  29%|██▉       | 2886/10000 [04:34<10:20, 11.46it/s, est. speed input: 3425.45 toks/s, output: 814.80 toks/s]
Processed prompts:  29%|██▉       | 2889/10000 [04:34<10:49, 10.94it/s, est. speed input: 3424.35 toks/s, output: 814.80 toks/s]
Processed prompts:  29%|██▉       | 2894/10000 [04:34<09:22, 12.62it/s, est. speed input: 3426.52 toks/s, output: 815.48 toks/s]
Processed prompts:  29%|██▉       | 2896/10000 [04:35<11:37, 10.18it/s, est. speed input: 3424.14 toks/s, output: 814.86 toks/s]
Processed prompts:  29%|██▉       | 2900/10000 [04:35<09:52, 11.99it/s, est. speed input: 3425.48 toks/s, output: 815.32 toks/s]
Processed prompts:  29%|██▉       | 2903/10000 [04:35<10:56, 10.81it/s, est. speed input: 3424.41 toks/s, output: 815.12 toks/s]
Processed prompts:  29%|██▉       | 2910/10000 [04:36<07:54, 14.95it/s, est. speed input: 3427.38 toks/s, output: 816.17 toks/s]
Processed prompts:  29%|██▉       | 2912/10000 [04:36<10:48, 10.93it/s, est. speed input: 3423.78 toks/s, output: 815.54 toks/s]
Processed prompts:  29%|██▉       | 2917/10000 [04:36<08:58, 13.14it/s, est. speed input: 3425.86 toks/s, output: 816.26 toks/s]
Processed prompts:  29%|██▉       | 2919/10000 [04:37<12:52,  9.16it/s, est. speed input: 3421.21 toks/s, output: 815.40 toks/s]
Processed prompts:  29%|██▉       | 2922/10000 [04:37<11:29, 10.27it/s, est. speed input: 3421.25 toks/s, output: 815.66 toks/s]
Processed prompts:  29%|██▉       | 2924/10000 [04:37<12:55,  9.13it/s, est. speed input: 3419.37 toks/s, output: 815.36 toks/s]
Processed prompts:  29%|██▉       | 2926/10000 [04:38<13:20,  8.84it/s, est. speed input: 3418.47 toks/s, output: 815.08 toks/s]
Processed prompts:  29%|██▉       | 2927/10000 [04:38<14:20,  8.22it/s, est. speed input: 3417.40 toks/s, output: 814.92 toks/s]
Processed prompts:  29%|██▉       | 2930/10000 [04:38<12:12,  9.65it/s, est. speed input: 3417.36 toks/s, output: 815.17 toks/s]
Processed prompts:  29%|██▉       | 2934/10000 [04:38<10:57, 10.74it/s, est. speed input: 3417.57 toks/s, output: 815.57 toks/s]
Processed prompts:  29%|██▉       | 2938/10000 [04:39<10:34, 11.14it/s, est. speed input: 3416.81 toks/s, output: 815.57 toks/s]
Processed prompts:  29%|██▉       | 2941/10000 [04:39<11:13, 10.49it/s, est. speed input: 3415.44 toks/s, output: 815.36 toks/s]
Processed prompts:  29%|██▉       | 2946/10000 [04:39<09:25, 12.47it/s, est. speed input: 3416.68 toks/s, output: 815.89 toks/s]
Processed prompts:  29%|██▉       | 2948/10000 [04:40<11:58,  9.82it/s, est. speed input: 3414.03 toks/s, output: 815.43 toks/s]
Processed prompts:  30%|██▉       | 2950/10000 [04:40<12:28,  9.42it/s, est. speed input: 3413.27 toks/s, output: 815.45 toks/s]
Processed prompts:  30%|██▉       | 2954/10000 [04:40<10:08, 11.58it/s, est. speed input: 3414.96 toks/s, output: 815.86 toks/s]
Processed prompts:  30%|██▉       | 2958/10000 [04:40<09:33, 12.29it/s, est. speed input: 3414.77 toks/s, output: 815.96 toks/s]
Processed prompts:  30%|██▉       | 2960/10000 [04:41<11:41, 10.03it/s, est. speed input: 3412.54 toks/s, output: 815.52 toks/s]
Processed prompts:  30%|██▉       | 2962/10000 [04:41<12:18,  9.53it/s, est. speed input: 3411.74 toks/s, output: 815.32 toks/s]
Processed prompts:  30%|██▉       | 2965/10000 [04:41<11:06, 10.56it/s, est. speed input: 3412.72 toks/s, output: 815.65 toks/s]
Processed prompts:  30%|██▉       | 2968/10000 [04:41<10:57, 10.69it/s, est. speed input: 3412.44 toks/s, output: 815.77 toks/s]
Processed prompts:  30%|██▉       | 2974/10000 [04:42<08:09, 14.34it/s, est. speed input: 3414.66 toks/s, output: 816.78 toks/s]
Processed prompts:  30%|██▉       | 2977/10000 [04:42<10:28, 11.18it/s, est. speed input: 3411.96 toks/s, output: 816.38 toks/s]
Processed prompts:  30%|██▉       | 2981/10000 [04:43<12:32,  9.32it/s, est. speed input: 3409.35 toks/s, output: 815.69 toks/s]
Processed prompts:  30%|██▉       | 2985/10000 [04:43<11:45,  9.94it/s, est. speed input: 3409.09 toks/s, output: 815.86 toks/s]
Processed prompts:  30%|██▉       | 2989/10000 [04:43<10:05, 11.57it/s, est. speed input: 3410.83 toks/s, output: 816.24 toks/s]
Processed prompts:  30%|██▉       | 2991/10000 [04:44<12:16,  9.52it/s, est. speed input: 3408.17 toks/s, output: 815.82 toks/s]
Processed prompts:  30%|███       | 3000/10000 [04:44<07:27, 15.66it/s, est. speed input: 3415.73 toks/s, output: 817.97 toks/s]
Processed prompts:  30%|███       | 3005/10000 [04:44<08:03, 14.46it/s, est. speed input: 3415.72 toks/s, output: 818.38 toks/s]
Processed prompts:  30%|███       | 3008/10000 [04:45<09:26, 12.34it/s, est. speed input: 3413.76 toks/s, output: 818.18 toks/s]
Processed prompts:  30%|███       | 3010/10000 [04:45<10:21, 11.25it/s, est. speed input: 3412.60 toks/s, output: 818.03 toks/s]
Processed prompts:  30%|███       | 3013/10000 [04:45<10:10, 11.45it/s, est. speed input: 3412.82 toks/s, output: 818.18 toks/s]
Processed prompts:  30%|███       | 3016/10000 [04:45<10:10, 11.44it/s, est. speed input: 3412.48 toks/s, output: 818.23 toks/s]
Processed prompts:  30%|███       | 3018/10000 [04:46<11:21, 10.25it/s, est. speed input: 3411.31 toks/s, output: 818.02 toks/s]
Processed prompts:  30%|███       | 3020/10000 [04:46<14:47,  7.87it/s, est. speed input: 3408.18 toks/s, output: 817.37 toks/s]
Processed prompts:  30%|███       | 3022/10000 [04:46<13:59,  8.31it/s, est. speed input: 3407.83 toks/s, output: 817.41 toks/s]
Processed prompts:  30%|███       | 3025/10000 [04:47<12:36,  9.22it/s, est. speed input: 3407.59 toks/s, output: 817.54 toks/s]
Processed prompts:  30%|███       | 3029/10000 [04:47<10:44, 10.81it/s, est. speed input: 3408.52 toks/s, output: 817.82 toks/s]
Processed prompts:  30%|███       | 3031/10000 [04:47<12:32,  9.26it/s, est. speed input: 3406.84 toks/s, output: 817.51 toks/s]
Processed prompts:  30%|███       | 3034/10000 [04:47<11:37,  9.98it/s, est. speed input: 3406.26 toks/s, output: 817.71 toks/s]
Processed prompts:  30%|███       | 3037/10000 [04:48<11:25, 10.16it/s, est. speed input: 3405.22 toks/s, output: 817.50 toks/s]
Processed prompts:  30%|███       | 3039/10000 [04:48<12:20,  9.40it/s, est. speed input: 3404.01 toks/s, output: 817.39 toks/s]
Processed prompts:  30%|███       | 3041/10000 [04:48<11:49,  9.81it/s, est. speed input: 3404.00 toks/s, output: 817.47 toks/s]
Processed prompts:  30%|███       | 3043/10000 [04:48<12:39,  9.16it/s, est. speed input: 3402.18 toks/s, output: 817.24 toks/s]
Processed prompts:  30%|███       | 3046/10000 [04:49<11:35, 10.00it/s, est. speed input: 3402.23 toks/s, output: 817.45 toks/s]
Processed prompts:  30%|███       | 3050/10000 [04:49<10:18, 11.24it/s, est. speed input: 3402.74 toks/s, output: 817.90 toks/s]
Processed prompts:  31%|███       | 3054/10000 [04:49<09:48, 11.80it/s, est. speed input: 3403.15 toks/s, output: 818.11 toks/s]
Processed prompts:  31%|███       | 3058/10000 [04:50<09:43, 11.90it/s, est. speed input: 3404.40 toks/s, output: 818.22 toks/s]
Processed prompts:  31%|███       | 3062/10000 [04:50<09:42, 11.92it/s, est. speed input: 3404.94 toks/s, output: 818.30 toks/s]
Processed prompts:  31%|███       | 3065/10000 [04:50<10:04, 11.47it/s, est. speed input: 3404.16 toks/s, output: 818.38 toks/s]
Processed prompts:  31%|███       | 3067/10000 [04:51<11:01, 10.48it/s, est. speed input: 3403.20 toks/s, output: 818.33 toks/s]
Processed prompts:  31%|███       | 3071/10000 [04:51<09:48, 11.77it/s, est. speed input: 3403.65 toks/s, output: 818.64 toks/s]
Processed prompts:  31%|███       | 3075/10000 [04:51<09:46, 11.80it/s, est. speed input: 3404.51 toks/s, output: 818.88 toks/s]
Processed prompts:  31%|███       | 3080/10000 [04:52<11:49,  9.75it/s, est. speed input: 3401.05 toks/s, output: 818.29 toks/s]
Processed prompts:  31%|███       | 3086/10000 [04:52<09:51, 11.70it/s, est. speed input: 3402.69 toks/s, output: 818.87 toks/s]
Processed prompts:  31%|███       | 3089/10000 [04:52<10:08, 11.36it/s, est. speed input: 3402.11 toks/s, output: 818.96 toks/s]
Processed prompts:  31%|███       | 3094/10000 [04:53<08:47, 13.08it/s, est. speed input: 3404.54 toks/s, output: 819.47 toks/s]
Processed prompts:  31%|███       | 3098/10000 [04:53<08:57, 12.83it/s, est. speed input: 3405.07 toks/s, output: 819.62 toks/s]
Processed prompts:  31%|███       | 3103/10000 [04:53<08:38, 13.31it/s, est. speed input: 3406.77 toks/s, output: 819.80 toks/s]
Processed prompts:  31%|███       | 3105/10000 [04:54<10:43, 10.71it/s, est. speed input: 3404.33 toks/s, output: 819.36 toks/s]
Processed prompts:  31%|███       | 3108/10000 [04:54<09:45, 11.78it/s, est. speed input: 3405.06 toks/s, output: 819.75 toks/s]
Processed prompts:  31%|███       | 3116/10000 [04:54<07:01, 16.34it/s, est. speed input: 3408.57 toks/s, output: 820.98 toks/s]
Processed prompts:  31%|███       | 3120/10000 [04:55<08:35, 13.35it/s, est. speed input: 3406.45 toks/s, output: 820.66 toks/s]
Processed prompts:  31%|███       | 3124/10000 [04:55<08:46, 13.07it/s, est. speed input: 3406.87 toks/s, output: 820.62 toks/s]
Processed prompts:  31%|███▏      | 3126/10000 [04:55<10:14, 11.19it/s, est. speed input: 3405.27 toks/s, output: 820.28 toks/s]
Processed prompts:  31%|███▏      | 3131/10000 [04:56<08:39, 13.22it/s, est. speed input: 3406.44 toks/s, output: 820.75 toks/s]
Processed prompts:  31%|███▏      | 3134/10000 [04:56<09:37, 11.89it/s, est. speed input: 3406.01 toks/s, output: 820.51 toks/s]
Processed prompts:  31%|███▏      | 3138/10000 [04:56<09:04, 12.61it/s, est. speed input: 3407.64 toks/s, output: 820.98 toks/s]
Processed prompts:  31%|███▏      | 3141/10000 [04:57<09:45, 11.71it/s, est. speed input: 3407.72 toks/s, output: 821.02 toks/s]
Processed prompts:  31%|███▏      | 3144/10000 [04:57<10:03, 11.37it/s, est. speed input: 3408.20 toks/s, output: 820.97 toks/s]
Processed prompts:  31%|███▏      | 3147/10000 [04:57<10:18, 11.09it/s, est. speed input: 3408.14 toks/s, output: 821.06 toks/s]
Processed prompts:  32%|███▏      | 3150/10000 [04:57<10:18, 11.08it/s, est. speed input: 3407.58 toks/s, output: 821.10 toks/s]
Processed prompts:  32%|███▏      | 3155/10000 [04:58<08:33, 13.33it/s, est. speed input: 3409.39 toks/s, output: 821.71 toks/s]
Processed prompts:  32%|███▏      | 3160/10000 [04:58<10:58, 10.38it/s, est. speed input: 3407.47 toks/s, output: 821.31 toks/s]
Processed prompts:  32%|███▏      | 3169/10000 [04:59<08:06, 14.03it/s, est. speed input: 3413.39 toks/s, output: 822.59 toks/s]
Processed prompts:  32%|███▏      | 3174/10000 [04:59<10:18, 11.03it/s, est. speed input: 3410.08 toks/s, output: 822.21 toks/s]
Processed prompts:  32%|███▏      | 3181/10000 [05:00<08:59, 12.63it/s, est. speed input: 3411.82 toks/s, output: 822.86 toks/s]
Processed prompts:  32%|███▏      | 3187/10000 [05:00<07:47, 14.58it/s, est. speed input: 3414.23 toks/s, output: 823.53 toks/s]
Processed prompts:  32%|███▏      | 3189/10000 [05:00<09:35, 11.83it/s, est. speed input: 3411.80 toks/s, output: 823.18 toks/s]
Processed prompts:  32%|███▏      | 3194/10000 [05:01<08:01, 14.14it/s, est. speed input: 3415.51 toks/s, output: 823.97 toks/s]
Processed prompts:  32%|███▏      | 3196/10000 [05:01<10:33, 10.75it/s, est. speed input: 3412.66 toks/s, output: 823.25 toks/s]
Processed prompts:  32%|███▏      | 3200/10000 [05:01<09:11, 12.32it/s, est. speed input: 3414.24 toks/s, output: 823.67 toks/s]
Processed prompts:  32%|███▏      | 3204/10000 [05:02<09:15, 12.24it/s, est. speed input: 3414.60 toks/s, output: 824.04 toks/s]
Processed prompts:  32%|███▏      | 3208/10000 [05:02<11:19,  9.99it/s, est. speed input: 3411.67 toks/s, output: 823.25 toks/s]
Processed prompts:  32%|███▏      | 3211/10000 [05:02<11:06, 10.19it/s, est. speed input: 3412.04 toks/s, output: 823.34 toks/s]
Processed prompts:  32%|███▏      | 3213/10000 [05:03<11:41,  9.67it/s, est. speed input: 3411.06 toks/s, output: 823.29 toks/s]
Processed prompts:  32%|███▏      | 3215/10000 [05:03<11:16, 10.03it/s, est. speed input: 3411.18 toks/s, output: 823.44 toks/s]
Processed prompts:  32%|███▏      | 3219/10000 [05:03<10:40, 10.58it/s, est. speed input: 3410.82 toks/s, output: 823.55 toks/s]
Processed prompts:  32%|███▏      | 3223/10000 [05:04<10:12, 11.06it/s, est. speed input: 3410.59 toks/s, output: 823.77 toks/s]
Processed prompts:  32%|███▏      | 3227/10000 [05:04<09:30, 11.88it/s, est. speed input: 3411.02 toks/s, output: 823.87 toks/s]
Processed prompts:  32%|███▏      | 3229/10000 [05:04<11:01, 10.23it/s, est. speed input: 3409.10 toks/s, output: 823.50 toks/s]
Processed prompts:  32%|███▏      | 3231/10000 [05:04<11:45,  9.59it/s, est. speed input: 3408.39 toks/s, output: 823.22 toks/s]
Processed prompts:  32%|███▏      | 3232/10000 [05:05<14:03,  8.02it/s, est. speed input: 3406.48 toks/s, output: 822.88 toks/s]
Processed prompts:  32%|███▏      | 3238/10000 [05:05<08:19, 13.54it/s, est. speed input: 3410.08 toks/s, output: 823.88 toks/s]
Processed prompts:  32%|███▏      | 3242/10000 [05:05<09:25, 11.94it/s, est. speed input: 3408.96 toks/s, output: 823.75 toks/s]
Processed prompts:  32%|███▏      | 3245/10000 [05:06<10:43, 10.51it/s, est. speed input: 3407.56 toks/s, output: 823.68 toks/s]
Processed prompts:  32%|███▏      | 3248/10000 [05:06<10:20, 10.88it/s, est. speed input: 3407.60 toks/s, output: 823.83 toks/s]
Processed prompts:  32%|███▎      | 3250/10000 [05:06<14:00,  8.03it/s, est. speed input: 3403.63 toks/s, output: 822.91 toks/s]
Processed prompts:  33%|███▎      | 3251/10000 [05:07<14:46,  7.61it/s, est. speed input: 3402.47 toks/s, output: 822.73 toks/s]
Processed prompts:  33%|███▎      | 3255/10000 [05:07<10:36, 10.60it/s, est. speed input: 3403.88 toks/s, output: 823.13 toks/s]
Processed prompts:  33%|███▎      | 3257/10000 [05:07<14:49,  7.58it/s, est. speed input: 3400.07 toks/s, output: 822.35 toks/s]
Processed prompts:  33%|███▎      | 3260/10000 [05:07<12:07,  9.27it/s, est. speed input: 3401.16 toks/s, output: 822.59 toks/s]
Processed prompts:  33%|███▎      | 3262/10000 [05:08<13:04,  8.58it/s, est. speed input: 3399.77 toks/s, output: 822.46 toks/s]
Processed prompts:  33%|███▎      | 3268/10000 [05:08<08:47, 12.76it/s, est. speed input: 3401.75 toks/s, output: 823.22 toks/s]
Processed prompts:  33%|███▎      | 3270/10000 [05:08<11:15,  9.96it/s, est. speed input: 3399.19 toks/s, output: 822.77 toks/s]
Processed prompts:  33%|███▎      | 3274/10000 [05:09<09:15, 12.11it/s, est. speed input: 3399.86 toks/s, output: 823.28 toks/s]
Processed prompts:  33%|███▎      | 3278/10000 [05:09<08:36, 13.01it/s, est. speed input: 3400.46 toks/s, output: 823.63 toks/s]
Processed prompts:  33%|███▎      | 3282/10000 [05:09<08:18, 13.47it/s, est. speed input: 3401.44 toks/s, output: 823.94 toks/s]
Processed prompts:  33%|███▎      | 3287/10000 [05:09<07:23, 15.15it/s, est. speed input: 3403.22 toks/s, output: 824.71 toks/s]
Processed prompts:  33%|███▎      | 3291/10000 [05:10<08:25, 13.27it/s, est. speed input: 3403.41 toks/s, output: 824.72 toks/s]
Processed prompts:  33%|███▎      | 3295/10000 [05:10<08:06, 13.78it/s, est. speed input: 3403.91 toks/s, output: 824.98 toks/s]
Processed prompts:  33%|███▎      | 3300/10000 [05:10<07:11, 15.52it/s, est. speed input: 3405.61 toks/s, output: 825.61 toks/s]
Processed prompts:  33%|███▎      | 3302/10000 [05:11<08:31, 13.10it/s, est. speed input: 3404.90 toks/s, output: 825.52 toks/s]
Processed prompts:  33%|███▎      | 3305/10000 [05:11<08:01, 13.90it/s, est. speed input: 3405.63 toks/s, output: 825.81 toks/s]
Processed prompts:  33%|███▎      | 3307/10000 [05:11<09:16, 12.02it/s, est. speed input: 3405.20 toks/s, output: 825.64 toks/s]
Processed prompts:  33%|███▎      | 3309/10000 [05:11<09:54, 11.25it/s, est. speed input: 3404.90 toks/s, output: 825.62 toks/s]
Processed prompts:  33%|███▎      | 3314/10000 [05:11<07:30, 14.85it/s, est. speed input: 3407.07 toks/s, output: 826.30 toks/s]
Processed prompts:  33%|███▎      | 3320/10000 [05:12<07:23, 15.08it/s, est. speed input: 3407.92 toks/s, output: 826.75 toks/s]
Processed prompts:  33%|███▎      | 3326/10000 [05:12<06:48, 16.34it/s, est. speed input: 3409.92 toks/s, output: 827.28 toks/s]
Processed prompts:  33%|███▎      | 3328/10000 [05:12<09:06, 12.21it/s, est. speed input: 3407.48 toks/s, output: 826.83 toks/s]
Processed prompts:  33%|███▎      | 3333/10000 [05:13<07:54, 14.04it/s, est. speed input: 3409.19 toks/s, output: 827.30 toks/s]
Processed prompts:  33%|███▎      | 3336/10000 [05:13<08:22, 13.27it/s, est. speed input: 3409.25 toks/s, output: 827.40 toks/s]
Processed prompts:  33%|███▎      | 3343/10000 [05:13<06:40, 16.60it/s, est. speed input: 3412.26 toks/s, output: 828.43 toks/s]
Processed prompts:  33%|███▎      | 3349/10000 [05:14<07:09, 15.48it/s, est. speed input: 3411.85 toks/s, output: 828.51 toks/s]
Processed prompts:  34%|███▎      | 3351/10000 [05:14<08:52, 12.48it/s, est. speed input: 3409.91 toks/s, output: 828.22 toks/s]
Processed prompts:  34%|███▎      | 3355/10000 [05:14<07:40, 14.43it/s, est. speed input: 3411.04 toks/s, output: 828.80 toks/s]
Processed prompts:  34%|███▎      | 3358/10000 [05:15<08:44, 12.67it/s, est. speed input: 3409.99 toks/s, output: 828.56 toks/s]
Processed prompts:  34%|███▎      | 3360/10000 [05:15<10:19, 10.71it/s, est. speed input: 3408.54 toks/s, output: 828.20 toks/s]
Processed prompts:  34%|███▎      | 3362/10000 [05:15<11:49,  9.35it/s, est. speed input: 3406.99 toks/s, output: 827.85 toks/s]
Processed prompts:  34%|███▎      | 3366/10000 [05:16<12:13,  9.05it/s, est. speed input: 3406.18 toks/s, output: 827.71 toks/s]
Processed prompts:  34%|███▍      | 3376/10000 [05:16<07:23, 14.93it/s, est. speed input: 3411.14 toks/s, output: 829.29 toks/s]
Processed prompts:  34%|███▍      | 3381/10000 [05:16<07:43, 14.28it/s, est. speed input: 3410.78 toks/s, output: 829.65 toks/s]
Processed prompts:  34%|███▍      | 3385/10000 [05:17<07:39, 14.39it/s, est. speed input: 3411.12 toks/s, output: 829.96 toks/s]
Processed prompts:  34%|███▍      | 3389/10000 [05:17<07:55, 13.89it/s, est. speed input: 3410.67 toks/s, output: 830.03 toks/s]
Processed prompts:  34%|███▍      | 3391/10000 [05:17<09:11, 11.98it/s, est. speed input: 3410.54 toks/s, output: 829.79 toks/s]
Processed prompts:  34%|███▍      | 3395/10000 [05:18<08:12, 13.42it/s, est. speed input: 3411.10 toks/s, output: 830.14 toks/s]
Processed prompts:  34%|███▍      | 3399/10000 [05:18<08:32, 12.88it/s, est. speed input: 3410.97 toks/s, output: 830.15 toks/s]
Processed prompts:  34%|███▍      | 3402/10000 [05:18<09:17, 11.84it/s, est. speed input: 3411.04 toks/s, output: 830.15 toks/s]
Processed prompts:  34%|███▍      | 3404/10000 [05:18<10:08, 10.84it/s, est. speed input: 3409.90 toks/s, output: 830.01 toks/s]
Processed prompts:  34%|███▍      | 3407/10000 [05:19<09:17, 11.83it/s, est. speed input: 3410.50 toks/s, output: 830.25 toks/s]
Processed prompts:  34%|███▍      | 3410/10000 [05:19<09:14, 11.88it/s, est. speed input: 3410.15 toks/s, output: 830.40 toks/s]
Processed prompts:  34%|███▍      | 3412/10000 [05:19<10:30, 10.44it/s, est. speed input: 3409.26 toks/s, output: 830.18 toks/s]
Processed prompts:  34%|███▍      | 3414/10000 [05:19<11:23,  9.64it/s, est. speed input: 3408.33 toks/s, output: 829.93 toks/s]
Processed prompts:  34%|███▍      | 3416/10000 [05:20<12:00,  9.14it/s, est. speed input: 3407.20 toks/s, output: 829.81 toks/s]
Processed prompts:  34%|███▍      | 3420/10000 [05:20<09:24, 11.66it/s, est. speed input: 3407.95 toks/s, output: 830.37 toks/s]
Processed prompts:  34%|███▍      | 3422/10000 [05:20<12:09,  9.02it/s, est. speed input: 3405.76 toks/s, output: 829.96 toks/s]
Processed prompts:  34%|███▍      | 3428/10000 [05:20<07:42, 14.21it/s, est. speed input: 3410.17 toks/s, output: 831.02 toks/s]
Processed prompts:  34%|███▍      | 3431/10000 [05:21<09:35, 11.42it/s, est. speed input: 3408.85 toks/s, output: 830.77 toks/s]
Processed prompts:  34%|███▍      | 3436/10000 [05:21<08:01, 13.63it/s, est. speed input: 3409.93 toks/s, output: 831.31 toks/s]
Processed prompts:  34%|███▍      | 3438/10000 [05:21<09:37, 11.37it/s, est. speed input: 3408.54 toks/s, output: 831.19 toks/s]
Processed prompts:  34%|███▍      | 3441/10000 [05:22<09:24, 11.63it/s, est. speed input: 3408.47 toks/s, output: 831.35 toks/s]
Processed prompts:  34%|███▍      | 3447/10000 [05:22<07:53, 13.83it/s, est. speed input: 3409.77 toks/s, output: 832.05 toks/s]
Processed prompts:  35%|███▍      | 3453/10000 [05:22<07:01, 15.53it/s, est. speed input: 3412.10 toks/s, output: 832.65 toks/s]
Processed prompts:  35%|███▍      | 3455/10000 [05:23<09:07, 11.95it/s, est. speed input: 3409.61 toks/s, output: 832.21 toks/s]
Processed prompts:  35%|███▍      | 3460/10000 [05:23<07:52, 13.84it/s, est. speed input: 3411.17 toks/s, output: 832.78 toks/s]
Processed prompts:  35%|███▍      | 3463/10000 [05:23<09:18, 11.71it/s, est. speed input: 3410.67 toks/s, output: 832.72 toks/s]
Processed prompts:  35%|███▍      | 3468/10000 [05:24<08:03, 13.51it/s, est. speed input: 3410.91 toks/s, output: 833.16 toks/s]
Processed prompts:  35%|███▍      | 3470/10000 [05:24<09:45, 11.15it/s, est. speed input: 3409.65 toks/s, output: 832.90 toks/s]
Processed prompts:  35%|███▍      | 3472/10000 [05:24<09:55, 10.96it/s, est. speed input: 3408.86 toks/s, output: 832.93 toks/s]
Processed prompts:  35%|███▍      | 3474/10000 [05:24<10:43, 10.14it/s, est. speed input: 3408.07 toks/s, output: 832.74 toks/s]
Processed prompts:  35%|███▍      | 3479/10000 [05:25<08:43, 12.47it/s, est. speed input: 3408.85 toks/s, output: 833.18 toks/s]
Processed prompts:  35%|███▍      | 3481/10000 [05:25<10:38, 10.20it/s, est. speed input: 3407.02 toks/s, output: 832.83 toks/s]
Processed prompts:  35%|███▍      | 3485/10000 [05:25<08:53, 12.22it/s, est. speed input: 3407.92 toks/s, output: 833.16 toks/s]
Processed prompts:  35%|███▍      | 3487/10000 [05:25<09:51, 11.02it/s, est. speed input: 3407.15 toks/s, output: 833.01 toks/s]
Processed prompts:  35%|███▍      | 3489/10000 [05:26<09:59, 10.85it/s, est. speed input: 3407.42 toks/s, output: 833.04 toks/s]
Processed prompts:  35%|███▍      | 3492/10000 [05:26<12:49,  8.45it/s, est. speed input: 3404.23 toks/s, output: 832.43 toks/s]
Processed prompts:  35%|███▌      | 3500/10000 [05:27<08:35, 12.61it/s, est. speed input: 3405.33 toks/s, output: 833.14 toks/s]
Processed prompts:  35%|███▌      | 3502/10000 [05:27<10:07, 10.70it/s, est. speed input: 3403.56 toks/s, output: 832.72 toks/s]
Processed prompts:  35%|███▌      | 3504/10000 [05:27<10:14, 10.57it/s, est. speed input: 3402.93 toks/s, output: 832.67 toks/s]
Processed prompts:  35%|███▌      | 3506/10000 [05:27<10:04, 10.74it/s, est. speed input: 3402.33 toks/s, output: 832.70 toks/s]
Processed prompts:  35%|███▌      | 3509/10000 [05:27<09:38, 11.23it/s, est. speed input: 3402.34 toks/s, output: 832.72 toks/s]
Processed prompts:  35%|███▌      | 3513/10000 [05:28<08:45, 12.34it/s, est. speed input: 3401.68 toks/s, output: 832.91 toks/s]
Processed prompts:  35%|███▌      | 3515/10000 [05:28<10:03, 10.75it/s, est. speed input: 3399.93 toks/s, output: 832.69 toks/s]
Processed prompts:  35%|███▌      | 3518/10000 [05:28<10:23, 10.40it/s, est. speed input: 3399.46 toks/s, output: 832.58 toks/s]
Processed prompts:  35%|███▌      | 3523/10000 [05:29<08:14, 13.09it/s, est. speed input: 3400.06 toks/s, output: 833.04 toks/s]
Processed prompts:  35%|███▌      | 3532/10000 [05:29<06:21, 16.95it/s, est. speed input: 3402.25 toks/s, output: 834.23 toks/s]
Processed prompts:  35%|███▌      | 3540/10000 [05:29<06:12, 17.33it/s, est. speed input: 3402.84 toks/s, output: 834.87 toks/s]
Processed prompts:  35%|███▌      | 3542/10000 [05:30<08:23, 12.82it/s, est. speed input: 3399.61 toks/s, output: 834.22 toks/s]
Processed prompts:  35%|███▌      | 3544/10000 [05:30<09:13, 11.67it/s, est. speed input: 3399.37 toks/s, output: 834.13 toks/s]
Processed prompts:  35%|███▌      | 3546/10000 [05:30<09:10, 11.72it/s, est. speed input: 3399.45 toks/s, output: 834.34 toks/s]
Processed prompts:  36%|███▌      | 3554/10000 [05:31<06:14, 17.21it/s, est. speed input: 3403.07 toks/s, output: 835.70 toks/s]
Processed prompts:  36%|███▌      | 3556/10000 [05:31<10:07, 10.61it/s, est. speed input: 3398.07 toks/s, output: 834.68 toks/s]
Processed prompts:  36%|███▌      | 3558/10000 [05:31<10:03, 10.67it/s, est. speed input: 3398.26 toks/s, output: 834.65 toks/s]
Processed prompts:  36%|███▌      | 3560/10000 [05:32<10:48,  9.93it/s, est. speed input: 3397.22 toks/s, output: 834.54 toks/s]
Processed prompts:  36%|███▌      | 3565/10000 [05:32<08:04, 13.27it/s, est. speed input: 3398.04 toks/s, output: 835.25 toks/s]
Processed prompts:  36%|███▌      | 3572/10000 [05:32<06:45, 15.84it/s, est. speed input: 3398.55 toks/s, output: 835.98 toks/s]
Processed prompts:  36%|███▌      | 3577/10000 [05:33<07:24, 14.45it/s, est. speed input: 3398.09 toks/s, output: 836.05 toks/s]
Processed prompts:  36%|███▌      | 3579/10000 [05:33<08:31, 12.56it/s, est. speed input: 3396.27 toks/s, output: 835.82 toks/s]
Processed prompts:  36%|███▌      | 3583/10000 [05:33<08:01, 13.33it/s, est. speed input: 3397.23 toks/s, output: 836.05 toks/s]
Processed prompts:  36%|███▌      | 3586/10000 [05:33<08:25, 12.70it/s, est. speed input: 3396.50 toks/s, output: 836.04 toks/s]
Processed prompts:  36%|███▌      | 3588/10000 [05:34<10:41, 10.00it/s, est. speed input: 3393.56 toks/s, output: 835.50 toks/s]
Processed prompts:  36%|███▌      | 3594/10000 [05:34<07:52, 13.55it/s, est. speed input: 3395.12 toks/s, output: 836.19 toks/s]
Processed prompts:  36%|███▌      | 3596/10000 [05:34<11:05,  9.62it/s, est. speed input: 3392.21 toks/s, output: 835.49 toks/s]
Processed prompts:  36%|███▌      | 3599/10000 [05:35<09:44, 10.95it/s, est. speed input: 3393.01 toks/s, output: 835.74 toks/s]
Processed prompts:  36%|███▌      | 3605/10000 [05:35<07:06, 14.99it/s, est. speed input: 3396.14 toks/s, output: 836.55 toks/s]
Processed prompts:  36%|███▌      | 3610/10000 [05:35<07:32, 14.13it/s, est. speed input: 3396.41 toks/s, output: 836.70 toks/s]
Processed prompts:  36%|███▌      | 3613/10000 [05:36<08:24, 12.66it/s, est. speed input: 3396.22 toks/s, output: 836.57 toks/s]
Processed prompts:  36%|███▌      | 3616/10000 [05:36<08:10, 13.01it/s, est. speed input: 3396.24 toks/s, output: 836.72 toks/s]
Processed prompts:  36%|███▌      | 3620/10000 [05:36<08:08, 13.07it/s, est. speed input: 3395.29 toks/s, output: 836.78 toks/s]
Processed prompts:  36%|███▌      | 3623/10000 [05:36<09:08, 11.62it/s, est. speed input: 3394.62 toks/s, output: 836.55 toks/s]
Processed prompts:  36%|███▋      | 3625/10000 [05:37<10:02, 10.59it/s, est. speed input: 3393.07 toks/s, output: 836.36 toks/s]
Processed prompts:  36%|███▋      | 3629/10000 [05:37<08:44, 12.14it/s, est. speed input: 3395.56 toks/s, output: 836.59 toks/s]
Processed prompts:  36%|███▋      | 3631/10000 [05:37<09:52, 10.76it/s, est. speed input: 3394.56 toks/s, output: 836.48 toks/s]
Processed prompts:  36%|███▋      | 3635/10000 [05:37<08:11, 12.96it/s, est. speed input: 3395.63 toks/s, output: 836.91 toks/s]
Processed prompts:  36%|███▋      | 3640/10000 [05:38<07:32, 14.06it/s, est. speed input: 3395.77 toks/s, output: 837.33 toks/s]
Processed prompts:  36%|███▋      | 3644/10000 [05:38<08:00, 13.22it/s, est. speed input: 3395.66 toks/s, output: 837.40 toks/s]
Processed prompts:  36%|███▋      | 3649/10000 [05:38<07:57, 13.31it/s, est. speed input: 3396.33 toks/s, output: 837.53 toks/s]
Processed prompts:  37%|███▋      | 3651/10000 [05:39<09:33, 11.07it/s, est. speed input: 3394.19 toks/s, output: 837.16 toks/s]
Processed prompts:  37%|███▋      | 3655/10000 [05:39<08:04, 13.10it/s, est. speed input: 3394.81 toks/s, output: 837.55 toks/s]
Processed prompts:  37%|███▋      | 3660/10000 [05:39<07:43, 13.69it/s, est. speed input: 3394.87 toks/s, output: 837.81 toks/s]
Processed prompts:  37%|███▋      | 3662/10000 [05:40<09:21, 11.28it/s, est. speed input: 3393.66 toks/s, output: 837.48 toks/s]
Processed prompts:  37%|███▋      | 3665/10000 [05:40<08:27, 12.48it/s, est. speed input: 3395.66 toks/s, output: 837.70 toks/s]
Processed prompts:  37%|███▋      | 3669/10000 [05:40<08:08, 12.96it/s, est. speed input: 3394.71 toks/s, output: 837.83 toks/s]
Processed prompts:  37%|███▋      | 3671/10000 [05:40<09:42, 10.86it/s, est. speed input: 3393.65 toks/s, output: 837.52 toks/s]
Processed prompts:  37%|███▋      | 3675/10000 [05:41<08:31, 12.37it/s, est. speed input: 3393.65 toks/s, output: 837.78 toks/s]
Processed prompts:  37%|███▋      | 3677/10000 [05:41<10:07, 10.41it/s, est. speed input: 3392.34 toks/s, output: 837.42 toks/s]
Processed prompts:  37%|███▋      | 3679/10000 [05:41<10:46,  9.78it/s, est. speed input: 3390.81 toks/s, output: 837.23 toks/s]
Processed prompts:  37%|███▋      | 3681/10000 [05:41<10:39,  9.88it/s, est. speed input: 3390.00 toks/s, output: 837.21 toks/s]
Processed prompts:  37%|███▋      | 3683/10000 [05:41<10:12, 10.32it/s, est. speed input: 3389.25 toks/s, output: 837.27 toks/s]
Processed prompts:  37%|███▋      | 3686/10000 [05:42<09:38, 10.92it/s, est. speed input: 3389.38 toks/s, output: 837.31 toks/s]
Processed prompts:  37%|███▋      | 3694/10000 [05:42<06:06, 17.19it/s, est. speed input: 3391.88 toks/s, output: 838.35 toks/s]
Processed prompts:  37%|███▋      | 3696/10000 [05:42<08:11, 12.82it/s, est. speed input: 3389.62 toks/s, output: 837.93 toks/s]
Processed prompts:  37%|███▋      | 3699/10000 [05:43<07:48, 13.45it/s, est. speed input: 3391.27 toks/s, output: 838.12 toks/s]
Processed prompts:  37%|███▋      | 3705/10000 [05:43<06:21, 16.50it/s, est. speed input: 3392.15 toks/s, output: 838.88 toks/s]
Processed prompts:  37%|███▋      | 3707/10000 [05:43<08:38, 12.13it/s, est. speed input: 3389.42 toks/s, output: 838.39 toks/s]
Processed prompts:  37%|███▋      | 3709/10000 [05:43<08:45, 11.98it/s, est. speed input: 3389.14 toks/s, output: 838.37 toks/s]
Processed prompts:  37%|███▋      | 3711/10000 [05:44<10:21, 10.11it/s, est. speed input: 3388.18 toks/s, output: 838.07 toks/s]
Processed prompts:  37%|███▋      | 3713/10000 [05:44<10:21, 10.11it/s, est. speed input: 3387.10 toks/s, output: 838.01 toks/s]
Processed prompts:  37%|███▋      | 3717/10000 [05:44<08:54, 11.75it/s, est. speed input: 3387.04 toks/s, output: 838.11 toks/s]
Processed prompts:  37%|███▋      | 3719/10000 [05:44<10:47,  9.71it/s, est. speed input: 3385.78 toks/s, output: 837.72 toks/s]
Processed prompts:  37%|███▋      | 3721/10000 [05:45<11:14,  9.31it/s, est. speed input: 3384.53 toks/s, output: 837.62 toks/s]
Processed prompts:  37%|███▋      | 3725/10000 [05:45<08:45, 11.94it/s, est. speed input: 3386.63 toks/s, output: 837.89 toks/s]
Processed prompts:  37%|███▋      | 3730/10000 [05:45<07:26, 14.04it/s, est. speed input: 3388.61 toks/s, output: 838.25 toks/s]
Processed prompts:  37%|███▋      | 3737/10000 [05:46<06:39, 15.67it/s, est. speed input: 3390.17 toks/s, output: 838.71 toks/s]
Processed prompts:  37%|███▋      | 3740/10000 [05:46<08:46, 11.88it/s, est. speed input: 3387.26 toks/s, output: 838.21 toks/s]
Processed prompts:  37%|███▋      | 3742/10000 [05:46<09:09, 11.38it/s, est. speed input: 3386.13 toks/s, output: 838.14 toks/s]
Processed prompts:  37%|███▋      | 3746/10000 [05:46<08:23, 12.41it/s, est. speed input: 3386.72 toks/s, output: 838.35 toks/s]
Processed prompts:  38%|███▊      | 3752/10000 [05:47<06:03, 17.18it/s, est. speed input: 3390.38 toks/s, output: 839.27 toks/s]
Processed prompts:  38%|███▊      | 3755/10000 [05:47<07:35, 13.71it/s, est. speed input: 3389.29 toks/s, output: 839.05 toks/s]
Processed prompts:  38%|███▊      | 3759/10000 [05:47<07:15, 14.32it/s, est. speed input: 3390.29 toks/s, output: 839.36 toks/s]
Processed prompts:  38%|███▊      | 3762/10000 [05:48<08:23, 12.39it/s, est. speed input: 3388.86 toks/s, output: 839.18 toks/s]
Processed prompts:  38%|███▊      | 3765/10000 [05:48<08:46, 11.85it/s, est. speed input: 3387.69 toks/s, output: 839.19 toks/s]
Processed prompts:  38%|███▊      | 3768/10000 [05:48<11:22,  9.13it/s, est. speed input: 3385.08 toks/s, output: 838.51 toks/s]
Processed prompts:  38%|███▊      | 3775/10000 [05:49<08:57, 11.58it/s, est. speed input: 3384.30 toks/s, output: 838.91 toks/s]
Processed prompts:  38%|███▊      | 3777/10000 [05:49<09:47, 10.59it/s, est. speed input: 3382.87 toks/s, output: 838.72 toks/s]
Processed prompts:  38%|███▊      | 3783/10000 [05:49<07:10, 14.43it/s, est. speed input: 3386.96 toks/s, output: 839.37 toks/s]
Processed prompts:  38%|███▊      | 3788/10000 [05:50<07:26, 13.90it/s, est. speed input: 3386.73 toks/s, output: 839.44 toks/s]
Processed prompts:  38%|███▊      | 3793/10000 [05:50<07:54, 13.08it/s, est. speed input: 3387.05 toks/s, output: 839.48 toks/s]
Processed prompts:  38%|███▊      | 3798/10000 [05:50<07:26, 13.88it/s, est. speed input: 3387.72 toks/s, output: 839.80 toks/s]
Processed prompts:  38%|███▊      | 3800/10000 [05:51<08:59, 11.50it/s, est. speed input: 3385.99 toks/s, output: 839.42 toks/s]
Processed prompts:  38%|███▊      | 3803/10000 [05:51<08:14, 12.53it/s, est. speed input: 3387.07 toks/s, output: 839.57 toks/s]
Processed prompts:  38%|███▊      | 3808/10000 [05:51<07:17, 14.15it/s, est. speed input: 3388.73 toks/s, output: 839.91 toks/s]
Processed prompts:  38%|███▊      | 3811/10000 [05:52<09:08, 11.29it/s, est. speed input: 3386.94 toks/s, output: 839.47 toks/s]
Processed prompts:  38%|███▊      | 3813/10000 [05:52<10:05, 10.23it/s, est. speed input: 3386.00 toks/s, output: 839.20 toks/s]
Processed prompts:  38%|███▊      | 3819/10000 [05:52<07:12, 14.29it/s, est. speed input: 3390.78 toks/s, output: 839.96 toks/s]
Processed prompts:  38%|███▊      | 3822/10000 [05:53<09:19, 11.05it/s, est. speed input: 3388.27 toks/s, output: 839.50 toks/s]
Processed prompts:  38%|███▊      | 3827/10000 [05:53<08:18, 12.39it/s, est. speed input: 3388.01 toks/s, output: 839.76 toks/s]
Processed prompts:  38%|███▊      | 3829/10000 [05:53<11:08,  9.23it/s, est. speed input: 3385.36 toks/s, output: 839.08 toks/s]
Processed prompts:  38%|███▊      | 3839/10000 [05:54<07:44, 13.26it/s, est. speed input: 3385.89 toks/s, output: 839.96 toks/s]
Processed prompts:  38%|███▊      | 3845/10000 [05:54<06:42, 15.28it/s, est. speed input: 3386.65 toks/s, output: 840.62 toks/s]
Processed prompts:  38%|███▊      | 3847/10000 [05:55<08:33, 11.99it/s, est. speed input: 3384.40 toks/s, output: 840.06 toks/s]
Processed prompts:  38%|███▊      | 3850/10000 [05:55<08:15, 12.41it/s, est. speed input: 3385.36 toks/s, output: 840.25 toks/s]
Processed prompts:  39%|███▊      | 3852/10000 [05:55<09:32, 10.74it/s, est. speed input: 3384.47 toks/s, output: 839.97 toks/s]
Processed prompts:  39%|███▊      | 3856/10000 [05:55<08:04, 12.69it/s, est. speed input: 3386.08 toks/s, output: 840.36 toks/s]
Processed prompts:  39%|███▊      | 3858/10000 [05:56<10:29,  9.75it/s, est. speed input: 3383.62 toks/s, output: 839.73 toks/s]
Processed prompts:  39%|███▊      | 3860/10000 [05:56<12:46,  8.02it/s, est. speed input: 3380.82 toks/s, output: 839.20 toks/s]
Processed prompts:  39%|███▊      | 3865/10000 [05:56<08:57, 11.41it/s, est. speed input: 3382.43 toks/s, output: 839.83 toks/s]
Processed prompts:  39%|███▊      | 3869/10000 [05:57<08:40, 11.77it/s, est. speed input: 3381.95 toks/s, output: 839.90 toks/s]
Processed prompts:  39%|███▊      | 3872/10000 [05:57<08:56, 11.43it/s, est. speed input: 3381.52 toks/s, output: 839.84 toks/s]
Processed prompts:  39%|███▉      | 3877/10000 [05:57<07:10, 14.22it/s, est. speed input: 3382.98 toks/s, output: 840.43 toks/s]
Processed prompts:  39%|███▉      | 3883/10000 [05:57<06:33, 15.55it/s, est. speed input: 3385.38 toks/s, output: 840.83 toks/s]
Processed prompts:  39%|███▉      | 3890/10000 [05:58<06:33, 15.54it/s, est. speed input: 3385.18 toks/s, output: 841.15 toks/s]
Processed prompts:  39%|███▉      | 3892/10000 [05:58<08:45, 11.62it/s, est. speed input: 3382.96 toks/s, output: 840.53 toks/s]
Processed prompts:  39%|███▉      | 3896/10000 [05:59<07:43, 13.18it/s, est. speed input: 3383.29 toks/s, output: 840.86 toks/s]
Processed prompts:  39%|███▉      | 3899/10000 [05:59<08:36, 11.80it/s, est. speed input: 3383.00 toks/s, output: 840.59 toks/s]
Processed prompts:  39%|███▉      | 3903/10000 [05:59<08:05, 12.55it/s, est. speed input: 3383.26 toks/s, output: 840.73 toks/s]
Processed prompts:  39%|███▉      | 3907/10000 [06:00<08:13, 12.35it/s, est. speed input: 3382.47 toks/s, output: 840.76 toks/s]
Processed prompts:  39%|███▉      | 3910/10000 [06:00<08:25, 12.05it/s, est. speed input: 3381.93 toks/s, output: 840.80 toks/s]
Processed prompts:  39%|███▉      | 3914/10000 [06:00<07:46, 13.04it/s, est. speed input: 3381.91 toks/s, output: 841.07 toks/s]
Processed prompts:  39%|███▉      | 3922/10000 [06:00<06:30, 15.58it/s, est. speed input: 3383.00 toks/s, output: 841.65 toks/s]
Processed prompts:  39%|███▉      | 3928/10000 [06:01<09:39, 10.48it/s, est. speed input: 3377.98 toks/s, output: 840.69 toks/s]
Processed prompts:  39%|███▉      | 3930/10000 [06:02<09:20, 10.82it/s, est. speed input: 3377.95 toks/s, output: 840.76 toks/s]
Processed prompts:  39%|███▉      | 3932/10000 [06:02<09:29, 10.66it/s, est. speed input: 3377.24 toks/s, output: 840.70 toks/s]
Processed prompts:  39%|███▉      | 3937/10000 [06:02<07:31, 13.42it/s, est. speed input: 3378.65 toks/s, output: 841.29 toks/s]
Processed prompts:  39%|███▉      | 3941/10000 [06:02<07:44, 13.03it/s, est. speed input: 3378.59 toks/s, output: 841.38 toks/s]
Processed prompts:  39%|███▉      | 3945/10000 [06:03<07:58, 12.64it/s, est. speed input: 3378.36 toks/s, output: 841.31 toks/s]
Processed prompts:  39%|███▉      | 3947/10000 [06:03<09:31, 10.59it/s, est. speed input: 3377.22 toks/s, output: 840.93 toks/s]
Processed prompts:  39%|███▉      | 3949/10000 [06:03<09:18, 10.82it/s, est. speed input: 3376.48 toks/s, output: 840.89 toks/s]
Processed prompts:  40%|███▉      | 3951/10000 [06:03<09:34, 10.53it/s, est. speed input: 3375.43 toks/s, output: 840.77 toks/s]
Processed prompts:  40%|███▉      | 3955/10000 [06:04<08:15, 12.21it/s, est. speed input: 3375.91 toks/s, output: 840.91 toks/s]
Processed prompts:  40%|███▉      | 3957/10000 [06:04<10:07,  9.96it/s, est. speed input: 3375.74 toks/s, output: 840.48 toks/s]
Processed prompts:  40%|███▉      | 3959/10000 [06:04<10:58,  9.17it/s, est. speed input: 3374.38 toks/s, output: 840.27 toks/s]
Processed prompts:  40%|███▉      | 3961/10000 [06:04<10:43,  9.39it/s, est. speed input: 3373.58 toks/s, output: 840.23 toks/s]
Processed prompts:  40%|███▉      | 3965/10000 [06:05<08:17, 12.14it/s, est. speed input: 3375.12 toks/s, output: 840.63 toks/s]
Processed prompts:  40%|███▉      | 3969/10000 [06:05<07:36, 13.22it/s, est. speed input: 3374.89 toks/s, output: 840.84 toks/s]
Processed prompts:  40%|███▉      | 3971/10000 [06:05<09:34, 10.49it/s, est. speed input: 3374.18 toks/s, output: 840.47 toks/s]
Processed prompts:  40%|███▉      | 3975/10000 [06:05<07:49, 12.84it/s, est. speed input: 3375.69 toks/s, output: 840.74 toks/s]
Processed prompts:  40%|███▉      | 3981/10000 [06:06<06:14, 16.05it/s, est. speed input: 3377.93 toks/s, output: 841.32 toks/s]
Processed prompts:  40%|███▉      | 3985/10000 [06:06<06:42, 14.96it/s, est. speed input: 3378.08 toks/s, output: 841.41 toks/s]
Processed prompts:  40%|███▉      | 3990/10000 [06:06<06:21, 15.74it/s, est. speed input: 3378.09 toks/s, output: 841.62 toks/s]
Processed prompts:  40%|███▉      | 3992/10000 [06:07<08:32, 11.73it/s, est. speed input: 3375.44 toks/s, output: 841.19 toks/s]
Processed prompts:  40%|███▉      | 3996/10000 [06:07<07:45, 12.91it/s, est. speed input: 3375.52 toks/s, output: 841.48 toks/s]
Processed prompts:  40%|████      | 4001/10000 [06:07<06:54, 14.48it/s, est. speed input: 3376.72 toks/s, output: 841.77 toks/s]
Processed prompts:  40%|████      | 4004/10000 [06:07<07:52, 12.69it/s, est. speed input: 3376.38 toks/s, output: 841.58 toks/s]
Processed prompts:  40%|████      | 4009/10000 [06:08<07:05, 14.07it/s, est. speed input: 3376.96 toks/s, output: 841.90 toks/s]
Processed prompts:  40%|████      | 4015/10000 [06:08<06:21, 15.69it/s, est. speed input: 3380.85 toks/s, output: 842.28 toks/s]
Processed prompts:  40%|████      | 4017/10000 [06:08<08:00, 12.46it/s, est. speed input: 3378.92 toks/s, output: 841.93 toks/s]
Processed prompts:  40%|████      | 4021/10000 [06:09<07:25, 13.42it/s, est. speed input: 3379.61 toks/s, output: 842.14 toks/s]
Processed prompts:  40%|████      | 4023/10000 [06:09<08:51, 11.25it/s, est. speed input: 3377.75 toks/s, output: 841.84 toks/s]
Processed prompts:  40%|████      | 4025/10000 [06:09<09:31, 10.46it/s, est. speed input: 3377.14 toks/s, output: 841.67 toks/s]
Processed prompts:  40%|████      | 4031/10000 [06:10<09:38, 10.31it/s, est. speed input: 3375.03 toks/s, output: 841.43 toks/s]
Processed prompts:  40%|████      | 4038/10000 [06:10<07:06, 13.99it/s, est. speed input: 3378.01 toks/s, output: 842.28 toks/s]
Processed prompts:  40%|████      | 4041/10000 [06:10<08:23, 11.84it/s, est. speed input: 3377.33 toks/s, output: 841.99 toks/s]
Processed prompts:  40%|████      | 4044/10000 [06:11<08:23, 11.82it/s, est. speed input: 3377.50 toks/s, output: 842.00 toks/s]
Processed prompts:  40%|████      | 4048/10000 [06:11<07:46, 12.75it/s, est. speed input: 3377.21 toks/s, output: 842.24 toks/s]
Processed prompts:  41%|████      | 4051/10000 [06:11<08:22, 11.84it/s, est. speed input: 3378.24 toks/s, output: 842.14 toks/s]
Processed prompts:  41%|████      | 4054/10000 [06:12<08:29, 11.67it/s, est. speed input: 3377.53 toks/s, output: 842.13 toks/s]
Processed prompts:  41%|████      | 4056/10000 [06:12<11:03,  8.96it/s, est. speed input: 3375.99 toks/s, output: 841.55 toks/s]
Processed prompts:  41%|████      | 4061/10000 [06:12<08:01, 12.33it/s, est. speed input: 3378.38 toks/s, output: 842.13 toks/s]
Processed prompts:  41%|████      | 4063/10000 [06:12<09:39, 10.24it/s, est. speed input: 3376.41 toks/s, output: 841.76 toks/s]
Processed prompts:  41%|████      | 4066/10000 [06:13<08:50, 11.18it/s, est. speed input: 3376.86 toks/s, output: 841.85 toks/s]
Processed prompts:  41%|████      | 4072/10000 [06:13<06:48, 14.50it/s, est. speed input: 3377.67 toks/s, output: 842.38 toks/s]
Processed prompts:  41%|████      | 4075/10000 [06:14<10:10,  9.71it/s, est. speed input: 3375.53 toks/s, output: 841.58 toks/s]
Processed prompts:  41%|████      | 4084/10000 [06:14<06:59, 14.10it/s, est. speed input: 3378.50 toks/s, output: 842.59 toks/s]
Processed prompts:  41%|████      | 4089/10000 [06:14<07:07, 13.82it/s, est. speed input: 3377.28 toks/s, output: 842.72 toks/s]
Processed prompts:  41%|████      | 4091/10000 [06:15<08:23, 11.72it/s, est. speed input: 3375.67 toks/s, output: 842.38 toks/s]
Processed prompts:  41%|████      | 4095/10000 [06:15<07:13, 13.63it/s, est. speed input: 3376.62 toks/s, output: 842.63 toks/s]
Processed prompts:  41%|████      | 4099/10000 [06:15<07:26, 13.21it/s, est. speed input: 3376.69 toks/s, output: 842.62 toks/s]
Processed prompts:  41%|████      | 4101/10000 [06:15<08:36, 11.42it/s, est. speed input: 3375.44 toks/s, output: 842.42 toks/s]
Processed prompts:  41%|████      | 4104/10000 [06:16<08:07, 12.09it/s, est. speed input: 3376.79 toks/s, output: 842.58 toks/s]
Processed prompts:  41%|████      | 4108/10000 [06:16<07:29, 13.11it/s, est. speed input: 3376.76 toks/s, output: 842.75 toks/s]
Processed prompts:  41%|████      | 4110/10000 [06:16<08:59, 10.92it/s, est. speed input: 3375.86 toks/s, output: 842.44 toks/s]
Processed prompts:  41%|████      | 4114/10000 [06:16<07:39, 12.81it/s, est. speed input: 3377.79 toks/s, output: 842.75 toks/s]
Processed prompts:  41%|████      | 4121/10000 [06:17<06:06, 16.05it/s, est. speed input: 3380.09 toks/s, output: 843.47 toks/s]
Processed prompts:  41%|████      | 4123/10000 [06:17<08:31, 11.49it/s, est. speed input: 3377.02 toks/s, output: 842.92 toks/s]
Processed prompts:  41%|████▏     | 4128/10000 [06:17<07:13, 13.54it/s, est. speed input: 3377.60 toks/s, output: 843.36 toks/s]
Processed prompts:  41%|████▏     | 4134/10000 [06:18<06:33, 14.91it/s, est. speed input: 3378.42 toks/s, output: 843.81 toks/s]
Processed prompts:  41%|████▏     | 4136/10000 [06:18<08:25, 11.61it/s, est. speed input: 3376.39 toks/s, output: 843.38 toks/s]
Processed prompts:  41%|████▏     | 4138/10000 [06:18<08:43, 11.20it/s, est. speed input: 3376.47 toks/s, output: 843.33 toks/s]
Processed prompts:  41%|████▏     | 4140/10000 [06:19<09:24, 10.39it/s, est. speed input: 3375.12 toks/s, output: 843.16 toks/s]
Processed prompts:  41%|████▏     | 4147/10000 [06:19<06:06, 15.98it/s, est. speed input: 3379.22 toks/s, output: 844.03 toks/s]
Processed prompts:  41%|████▏     | 4149/10000 [06:19<08:42, 11.20it/s, est. speed input: 3376.97 toks/s, output: 843.45 toks/s]
Processed prompts:  42%|████▏     | 4151/10000 [06:19<09:01, 10.80it/s, est. speed input: 3376.97 toks/s, output: 843.36 toks/s]
Processed prompts:  42%|████▏     | 4153/10000 [06:20<08:52, 10.98it/s, est. speed input: 3376.41 toks/s, output: 843.37 toks/s]
Processed prompts:  42%|████▏     | 4159/10000 [06:20<06:12, 15.68it/s, est. speed input: 3378.64 toks/s, output: 844.03 toks/s]
Processed prompts:  42%|████▏     | 4164/10000 [06:20<06:52, 14.16it/s, est. speed input: 3379.22 toks/s, output: 844.06 toks/s]
Processed prompts:  42%|████▏     | 4168/10000 [06:21<07:09, 13.57it/s, est. speed input: 3381.42 toks/s, output: 844.07 toks/s]
Processed prompts:  42%|████▏     | 4171/10000 [06:21<07:42, 12.61it/s, est. speed input: 3381.36 toks/s, output: 844.01 toks/s]
Processed prompts:  42%|████▏     | 4173/10000 [06:21<09:29, 10.24it/s, est. speed input: 3379.59 toks/s, output: 843.58 toks/s]
Processed prompts:  42%|████▏     | 4175/10000 [06:21<09:03, 10.71it/s, est. speed input: 3379.35 toks/s, output: 843.68 toks/s]
Processed prompts:  42%|████▏     | 4180/10000 [06:22<06:22, 15.23it/s, est. speed input: 3381.28 toks/s, output: 844.38 toks/s]
Processed prompts:  42%|████▏     | 4185/10000 [06:22<06:23, 15.18it/s, est. speed input: 3380.67 toks/s, output: 844.64 toks/s]
Processed prompts:  42%|████▏     | 4189/10000 [06:22<06:48, 14.21it/s, est. speed input: 3380.56 toks/s, output: 844.74 toks/s]
Processed prompts:  42%|████▏     | 4192/10000 [06:22<07:21, 13.15it/s, est. speed input: 3381.04 toks/s, output: 844.68 toks/s]
Processed prompts:  42%|████▏     | 4194/10000 [06:23<08:31, 11.35it/s, est. speed input: 3380.26 toks/s, output: 844.48 toks/s]
Processed prompts:  42%|████▏     | 4199/10000 [06:23<06:45, 14.30it/s, est. speed input: 3382.25 toks/s, output: 844.93 toks/s]
Processed prompts:  42%|████▏     | 4204/10000 [06:23<06:35, 14.65it/s, est. speed input: 3382.96 toks/s, output: 845.10 toks/s]
Processed prompts:  42%|████▏     | 4206/10000 [06:24<08:06, 11.91it/s, est. speed input: 3381.20 toks/s, output: 844.77 toks/s]
Processed prompts:  42%|████▏     | 4208/10000 [06:24<08:09, 11.83it/s, est. speed input: 3381.03 toks/s, output: 844.77 toks/s]
Processed prompts:  42%|████▏     | 4211/10000 [06:24<08:01, 12.02it/s, est. speed input: 3380.16 toks/s, output: 844.81 toks/s]
Processed prompts:  42%|████▏     | 4215/10000 [06:24<07:37, 12.63it/s, est. speed input: 3379.70 toks/s, output: 844.93 toks/s]
Processed prompts:  42%|████▏     | 4219/10000 [06:25<07:33, 12.75it/s, est. speed input: 3379.95 toks/s, output: 844.99 toks/s]
Processed prompts:  42%|████▏     | 4222/10000 [06:25<08:05, 11.89it/s, est. speed input: 3380.07 toks/s, output: 844.91 toks/s]
Processed prompts:  42%|████▏     | 4226/10000 [06:25<07:30, 12.81it/s, est. speed input: 3379.79 toks/s, output: 845.12 toks/s]
Processed prompts:  42%|████▏     | 4231/10000 [06:25<06:45, 14.24it/s, est. speed input: 3381.67 toks/s, output: 845.44 toks/s]
Processed prompts:  42%|████▏     | 4233/10000 [06:26<08:24, 11.42it/s, est. speed input: 3379.77 toks/s, output: 845.06 toks/s]
Processed prompts:  42%|████▏     | 4238/10000 [06:26<06:56, 13.85it/s, est. speed input: 3381.13 toks/s, output: 845.45 toks/s]
Processed prompts:  42%|████▏     | 4240/10000 [06:26<08:36, 11.15it/s, est. speed input: 3379.63 toks/s, output: 845.11 toks/s]
Processed prompts:  42%|████▏     | 4244/10000 [06:27<07:24, 12.95it/s, est. speed input: 3380.07 toks/s, output: 845.42 toks/s]
Processed prompts:  42%|████▎     | 4250/10000 [06:27<06:26, 14.86it/s, est. speed input: 3380.54 toks/s, output: 845.86 toks/s]
Processed prompts:  43%|████▎     | 4254/10000 [06:27<07:16, 13.17it/s, est. speed input: 3379.38 toks/s, output: 845.74 toks/s]
Processed prompts:  43%|████▎     | 4259/10000 [06:28<06:39, 14.37it/s, est. speed input: 3380.27 toks/s, output: 846.14 toks/s]
Processed prompts:  43%|████▎     | 4263/10000 [06:28<07:00, 13.63it/s, est. speed input: 3379.52 toks/s, output: 846.13 toks/s]
Processed prompts:  43%|████▎     | 4267/10000 [06:28<06:57, 13.73it/s, est. speed input: 3379.17 toks/s, output: 846.23 toks/s]
Processed prompts:  43%|████▎     | 4271/10000 [06:29<07:06, 13.44it/s, est. speed input: 3378.98 toks/s, output: 846.29 toks/s]
Processed prompts:  43%|████▎     | 4274/10000 [06:29<07:35, 12.58it/s, est. speed input: 3378.62 toks/s, output: 846.19 toks/s]
Processed prompts:  43%|████▎     | 4277/10000 [06:29<07:43, 12.35it/s, est. speed input: 3378.20 toks/s, output: 846.23 toks/s]
Processed prompts:  43%|████▎     | 4282/10000 [06:29<06:40, 14.27it/s, est. speed input: 3378.64 toks/s, output: 846.52 toks/s]
Processed prompts:  43%|████▎     | 4285/10000 [06:30<09:38,  9.87it/s, est. speed input: 3375.62 toks/s, output: 845.71 toks/s]
Processed prompts:  43%|████▎     | 4292/10000 [06:30<07:31, 12.65it/s, est. speed input: 3377.19 toks/s, output: 846.33 toks/s]
Processed prompts:  43%|████▎     | 4294/10000 [06:31<08:25, 11.28it/s, est. speed input: 3375.99 toks/s, output: 846.12 toks/s]
Processed prompts:  43%|████▎     | 4299/10000 [06:31<06:51, 13.85it/s, est. speed input: 3377.23 toks/s, output: 846.46 toks/s]
Processed prompts:  43%|████▎     | 4301/10000 [06:31<08:08, 11.67it/s, est. speed input: 3375.87 toks/s, output: 846.17 toks/s]
Processed prompts:  43%|████▎     | 4303/10000 [06:31<08:09, 11.63it/s, est. speed input: 3375.41 toks/s, output: 846.17 toks/s]
Processed prompts:  43%|████▎     | 4307/10000 [06:32<07:30, 12.64it/s, est. speed input: 3375.22 toks/s, output: 846.32 toks/s]
Processed prompts:  43%|████▎     | 4311/10000 [06:32<09:49,  9.65it/s, est. speed input: 3372.74 toks/s, output: 845.75 toks/s]
Processed prompts:  43%|████▎     | 4317/10000 [06:32<08:03, 11.76it/s, est. speed input: 3373.43 toks/s, output: 846.16 toks/s]
Processed prompts:  43%|████▎     | 4320/10000 [06:33<08:08, 11.63it/s, est. speed input: 3372.86 toks/s, output: 846.09 toks/s]
Processed prompts:  43%|████▎     | 4324/10000 [06:33<07:35, 12.47it/s, est. speed input: 3373.15 toks/s, output: 846.19 toks/s]
Processed prompts:  43%|████▎     | 4327/10000 [06:33<07:55, 11.92it/s, est. speed input: 3372.87 toks/s, output: 846.11 toks/s]
Processed prompts:  43%|████▎     | 4329/10000 [06:34<08:49, 10.71it/s, est. speed input: 3371.89 toks/s, output: 845.94 toks/s]
Processed prompts:  43%|████▎     | 4334/10000 [06:34<06:52, 13.73it/s, est. speed input: 3373.37 toks/s, output: 846.43 toks/s]
Processed prompts:  43%|████▎     | 4338/10000 [06:34<07:07, 13.26it/s, est. speed input: 3373.16 toks/s, output: 846.50 toks/s]
Processed prompts:  43%|████▎     | 4342/10000 [06:34<07:09, 13.16it/s, est. speed input: 3373.37 toks/s, output: 846.57 toks/s]
Processed prompts:  43%|████▎     | 4344/10000 [06:35<08:28, 11.12it/s, est. speed input: 3372.02 toks/s, output: 846.29 toks/s]
Processed prompts:  43%|████▎     | 4348/10000 [06:35<07:16, 12.95it/s, est. speed input: 3372.85 toks/s, output: 846.57 toks/s]
Processed prompts:  44%|████▎     | 4355/10000 [06:35<05:41, 16.53it/s, est. speed input: 3374.82 toks/s, output: 847.17 toks/s]
Processed prompts:  44%|████▎     | 4358/10000 [06:36<07:13, 13.02it/s, est. speed input: 3373.36 toks/s, output: 846.81 toks/s]
Processed prompts:  44%|████▎     | 4361/10000 [06:36<07:27, 12.59it/s, est. speed input: 3373.19 toks/s, output: 846.79 toks/s]
Processed prompts:  44%|████▎     | 4365/10000 [06:36<07:05, 13.24it/s, est. speed input: 3373.37 toks/s, output: 846.93 toks/s]
Processed prompts:  44%|████▎     | 4370/10000 [06:36<06:26, 14.56it/s, est. speed input: 3374.06 toks/s, output: 847.21 toks/s]
Processed prompts:  44%|████▎     | 4372/10000 [06:37<08:02, 11.66it/s, est. speed input: 3372.59 toks/s, output: 846.94 toks/s]
Processed prompts:  44%|████▍     | 4378/10000 [06:37<05:56, 15.76it/s, est. speed input: 3375.37 toks/s, output: 847.63 toks/s]
Processed prompts:  44%|████▍     | 4381/10000 [06:37<07:31, 12.46it/s, est. speed input: 3373.79 toks/s, output: 847.28 toks/s]
Processed prompts:  44%|████▍     | 4383/10000 [06:38<08:23, 11.16it/s, est. speed input: 3372.82 toks/s, output: 847.09 toks/s]
Processed prompts:  44%|████▍     | 4385/10000 [06:38<08:22, 11.17it/s, est. speed input: 3372.67 toks/s, output: 847.10 toks/s]
Processed prompts:  44%|████▍     | 4389/10000 [06:38<07:37, 12.28it/s, est. speed input: 3373.02 toks/s, output: 847.26 toks/s]
Processed prompts:  44%|████▍     | 4393/10000 [06:38<07:17, 12.82it/s, est. speed input: 3373.27 toks/s, output: 847.39 toks/s]
Processed prompts:  44%|████▍     | 4396/10000 [06:39<07:35, 12.31it/s, est. speed input: 3372.72 toks/s, output: 847.36 toks/s]
Processed prompts:  44%|████▍     | 4398/10000 [06:39<08:45, 10.65it/s, est. speed input: 3371.43 toks/s, output: 847.10 toks/s]
Processed prompts:  44%|████▍     | 4401/10000 [06:39<07:45, 12.02it/s, est. speed input: 3371.75 toks/s, output: 847.26 toks/s]
Processed prompts:  44%|████▍     | 4405/10000 [06:39<07:20, 12.70it/s, est. speed input: 3371.96 toks/s, output: 847.37 toks/s]
Processed prompts:  44%|████▍     | 4408/10000 [06:40<07:44, 12.05it/s, est. speed input: 3371.46 toks/s, output: 847.29 toks/s]
Processed prompts:  44%|████▍     | 4413/10000 [06:40<06:34, 14.15it/s, est. speed input: 3372.73 toks/s, output: 847.65 toks/s]
Processed prompts:  44%|████▍     | 4416/10000 [06:40<07:32, 12.35it/s, est. speed input: 3371.76 toks/s, output: 847.52 toks/s]
Processed prompts:  44%|████▍     | 4421/10000 [06:41<06:34, 14.15it/s, est. speed input: 3372.63 toks/s, output: 847.89 toks/s]
Processed prompts:  44%|████▍     | 4425/10000 [06:41<06:59, 13.29it/s, est. speed input: 3372.05 toks/s, output: 847.89 toks/s]
Processed prompts:  44%|████▍     | 4428/10000 [06:41<07:35, 12.24it/s, est. speed input: 3371.25 toks/s, output: 847.77 toks/s]
Processed prompts:  44%|████▍     | 4432/10000 [06:41<07:07, 13.03it/s, est. speed input: 3371.58 toks/s, output: 848.01 toks/s]
Processed prompts:  44%|████▍     | 4435/10000 [06:42<07:35, 12.21it/s, est. speed input: 3371.15 toks/s, output: 847.94 toks/s]
Processed prompts:  44%|████▍     | 4438/10000 [06:42<07:47, 11.91it/s, est. speed input: 3370.60 toks/s, output: 847.89 toks/s]
Processed prompts:  44%|████▍     | 4440/10000 [06:42<08:47, 10.55it/s, est. speed input: 3369.62 toks/s, output: 847.65 toks/s]
Processed prompts:  44%|████▍     | 4442/10000 [06:42<09:03, 10.24it/s, est. speed input: 3369.17 toks/s, output: 847.54 toks/s]
Processed prompts:  44%|████▍     | 4444/10000 [06:43<09:09, 10.11it/s, est. speed input: 3368.67 toks/s, output: 847.44 toks/s]
Processed prompts:  44%|████▍     | 4446/10000 [06:43<09:45,  9.49it/s, est. speed input: 3367.99 toks/s, output: 847.41 toks/s]
Processed prompts:  45%|████▍     | 4452/10000 [06:43<06:01, 15.36it/s, est. speed input: 3370.45 toks/s, output: 848.13 toks/s]
Processed prompts:  45%|████▍     | 4457/10000 [06:44<06:45, 13.67it/s, est. speed input: 3369.86 toks/s, output: 848.10 toks/s]
Processed prompts:  45%|████▍     | 4459/10000 [06:44<08:40, 10.65it/s, est. speed input: 3367.85 toks/s, output: 847.71 toks/s]
Processed prompts:  45%|████▍     | 4461/10000 [06:44<09:14,  9.99it/s, est. speed input: 3367.19 toks/s, output: 847.55 toks/s]
Processed prompts:  45%|████▍     | 4466/10000 [06:44<06:58, 13.23it/s, est. speed input: 3368.80 toks/s, output: 848.05 toks/s]
Processed prompts:  45%|████▍     | 4470/10000 [06:45<07:14, 12.73it/s, est. speed input: 3368.64 toks/s, output: 848.12 toks/s]
Processed prompts:  45%|████▍     | 4472/10000 [06:45<08:22, 11.00it/s, est. speed input: 3367.55 toks/s, output: 847.85 toks/s]
Processed prompts:  45%|████▍     | 4474/10000 [06:45<10:31,  8.76it/s, est. speed input: 3365.44 toks/s, output: 847.39 toks/s]
Processed prompts:  45%|████▍     | 4478/10000 [06:46<08:11, 11.25it/s, est. speed input: 3366.61 toks/s, output: 847.73 toks/s]
Processed prompts:  45%|████▍     | 4485/10000 [06:46<06:09, 14.91it/s, est. speed input: 3368.53 toks/s, output: 848.36 toks/s]
Processed prompts:  45%|████▍     | 4488/10000 [06:46<07:47, 11.80it/s, est. speed input: 3366.88 toks/s, output: 847.96 toks/s]
Processed prompts:  45%|████▍     | 4495/10000 [06:47<05:55, 15.50it/s, est. speed input: 3368.88 toks/s, output: 848.65 toks/s]
Processed prompts:  45%|████▌     | 4500/10000 [06:47<06:32, 14.02it/s, est. speed input: 3368.65 toks/s, output: 848.67 toks/s]
Processed prompts:  45%|████▌     | 4507/10000 [06:47<05:40, 16.13it/s, est. speed input: 3370.74 toks/s, output: 849.22 toks/s]
Processed prompts:  45%|████▌     | 4512/10000 [06:48<06:10, 14.83it/s, est. speed input: 3370.47 toks/s, output: 849.24 toks/s]
Processed prompts:  45%|████▌     | 4514/10000 [06:48<07:24, 12.35it/s, est. speed input: 3369.07 toks/s, output: 848.92 toks/s]
Processed prompts:  45%|████▌     | 4518/10000 [06:48<06:27, 14.14it/s, est. speed input: 3369.95 toks/s, output: 849.24 toks/s]
Processed prompts:  45%|████▌     | 4523/10000 [06:49<06:21, 14.37it/s, est. speed input: 3370.23 toks/s, output: 849.45 toks/s]
Processed prompts:  45%|████▌     | 4525/10000 [06:49<07:47, 11.72it/s, est. speed input: 3368.75 toks/s, output: 849.11 toks/s]
Processed prompts:  45%|████▌     | 4527/10000 [06:49<07:48, 11.67it/s, est. speed input: 3368.59 toks/s, output: 849.18 toks/s]
Processed prompts:  45%|████▌     | 4531/10000 [06:49<06:49, 13.36it/s, est. speed input: 3369.33 toks/s, output: 849.44 toks/s]
Processed prompts:  45%|████▌     | 4535/10000 [06:50<07:00, 12.99it/s, est. speed input: 3369.39 toks/s, output: 849.55 toks/s]
Processed prompts:  45%|████▌     | 4537/10000 [06:50<08:07, 11.20it/s, est. speed input: 3368.42 toks/s, output: 849.34 toks/s]
Processed prompts:  45%|████▌     | 4539/10000 [06:50<08:26, 10.79it/s, est. speed input: 3367.77 toks/s, output: 849.28 toks/s]
Processed prompts:  45%|████▌     | 4542/10000 [06:50<08:10, 11.12it/s, est. speed input: 3367.50 toks/s, output: 849.26 toks/s]
Processed prompts:  45%|████▌     | 4545/10000 [06:51<08:01, 11.33it/s, est. speed input: 3367.50 toks/s, output: 849.21 toks/s]
Processed prompts:  45%|████▌     | 4548/10000 [06:51<08:04, 11.26it/s, est. speed input: 3367.35 toks/s, output: 849.19 toks/s]
Processed prompts:  46%|████▌     | 4550/10000 [06:51<09:00, 10.09it/s, est. speed input: 3366.66 toks/s, output: 849.03 toks/s]
Processed prompts:  46%|████▌     | 4553/10000 [06:51<08:06, 11.19it/s, est. speed input: 3366.90 toks/s, output: 849.15 toks/s]
Processed prompts:  46%|████▌     | 4558/10000 [06:52<06:37, 13.68it/s, est. speed input: 3368.09 toks/s, output: 849.49 toks/s]
Processed prompts:  46%|████▌     | 4560/10000 [06:52<08:15, 10.97it/s, est. speed input: 3366.60 toks/s, output: 849.18 toks/s]
Processed prompts:  46%|████▌     | 4563/10000 [06:52<07:41, 11.77it/s, est. speed input: 3366.37 toks/s, output: 849.29 toks/s]
Processed prompts:  46%|████▌     | 4566/10000 [06:52<07:48, 11.59it/s, est. speed input: 3366.32 toks/s, output: 849.27 toks/s]
Processed prompts:  46%|████▌     | 4568/10000 [06:53<09:21,  9.67it/s, est. speed input: 3364.82 toks/s, output: 848.97 toks/s]
Processed prompts:  46%|████▌     | 4579/10000 [06:53<05:25, 16.64it/s, est. speed input: 3368.72 toks/s, output: 850.16 toks/s]
Processed prompts:  46%|████▌     | 4582/10000 [06:54<06:39, 13.56it/s, est. speed input: 3367.56 toks/s, output: 849.92 toks/s]
Processed prompts:  46%|████▌     | 4586/10000 [06:54<06:29, 13.91it/s, est. speed input: 3367.70 toks/s, output: 850.09 toks/s]
Processed prompts:  46%|████▌     | 4589/10000 [06:54<07:05, 12.71it/s, est. speed input: 3367.03 toks/s, output: 849.98 toks/s]
Processed prompts:  46%|████▌     | 4591/10000 [06:54<07:54, 11.40it/s, est. speed input: 3366.08 toks/s, output: 849.79 toks/s]
Processed prompts:  46%|████▌     | 4593/10000 [06:55<08:13, 10.95it/s, est. speed input: 3365.77 toks/s, output: 849.71 toks/s]
Processed prompts:  46%|████▌     | 4596/10000 [06:55<07:19, 12.28it/s, est. speed input: 3366.23 toks/s, output: 849.86 toks/s]
Processed prompts:  46%|████▌     | 4601/10000 [06:55<06:35, 13.66it/s, est. speed input: 3366.60 toks/s, output: 850.06 toks/s]
Processed prompts:  46%|████▌     | 4603/10000 [06:55<08:08, 11.05it/s, est. speed input: 3365.21 toks/s, output: 849.79 toks/s]
Processed prompts:  46%|████▌     | 4606/10000 [06:56<07:53, 11.39it/s, est. speed input: 3365.06 toks/s, output: 849.81 toks/s]
Processed prompts:  46%|████▌     | 4611/10000 [06:56<06:38, 13.51it/s, est. speed input: 3365.52 toks/s, output: 850.17 toks/s]
Processed prompts:  46%|████▌     | 4615/10000 [06:56<06:55, 12.97it/s, est. speed input: 3365.44 toks/s, output: 850.15 toks/s]
Processed prompts:  46%|████▌     | 4618/10000 [06:57<07:28, 12.00it/s, est. speed input: 3364.70 toks/s, output: 850.06 toks/s]
Processed prompts:  46%|████▌     | 4620/10000 [06:57<08:20, 10.75it/s, est. speed input: 3364.03 toks/s, output: 849.88 toks/s]
Processed prompts:  46%|████▋     | 4625/10000 [06:57<06:41, 13.40it/s, est. speed input: 3365.05 toks/s, output: 850.23 toks/s]
Processed prompts:  46%|████▋     | 4627/10000 [06:57<08:37, 10.39it/s, est. speed input: 3363.40 toks/s, output: 849.84 toks/s]
Processed prompts:  46%|████▋     | 4630/10000 [06:58<07:56, 11.28it/s, est. speed input: 3363.55 toks/s, output: 849.90 toks/s]
Processed prompts:  46%|████▋     | 4632/10000 [06:58<08:46, 10.20it/s, est. speed input: 3362.80 toks/s, output: 849.73 toks/s]
Processed prompts:  46%|████▋     | 4634/10000 [06:58<08:55, 10.03it/s, est. speed input: 3362.42 toks/s, output: 849.72 toks/s]
Processed prompts:  46%|████▋     | 4637/10000 [06:58<07:47, 11.47it/s, est. speed input: 3362.85 toks/s, output: 849.84 toks/s]
Processed prompts:  46%|████▋     | 4645/10000 [06:59<05:14, 17.04it/s, est. speed input: 3365.69 toks/s, output: 850.60 toks/s]
Processed prompts:  46%|████▋     | 4648/10000 [06:59<06:57, 12.83it/s, est. speed input: 3364.33 toks/s, output: 850.28 toks/s]
Processed prompts:  47%|████▋     | 4652/10000 [06:59<06:38, 13.41it/s, est. speed input: 3364.56 toks/s, output: 850.44 toks/s]
Processed prompts:  47%|████▋     | 4654/10000 [07:00<08:02, 11.07it/s, est. speed input: 3362.98 toks/s, output: 850.14 toks/s]
Processed prompts:  47%|████▋     | 4657/10000 [07:00<07:46, 11.46it/s, est. speed input: 3363.09 toks/s, output: 850.19 toks/s]
Processed prompts:  47%|████▋     | 4662/10000 [07:00<06:30, 13.67it/s, est. speed input: 3364.27 toks/s, output: 850.52 toks/s]
Processed prompts:  47%|████▋     | 4664/10000 [07:01<09:25,  9.44it/s, est. speed input: 3361.52 toks/s, output: 849.95 toks/s]
Processed prompts:  47%|████▋     | 4672/10000 [07:01<05:39, 15.71it/s, est. speed input: 3364.90 toks/s, output: 850.89 toks/s]
Processed prompts:  47%|████▋     | 4675/10000 [07:01<07:11, 12.33it/s, est. speed input: 3363.06 toks/s, output: 850.55 toks/s]
Processed prompts:  47%|████▋     | 4680/10000 [07:01<05:54, 15.02it/s, est. speed input: 3364.64 toks/s, output: 851.00 toks/s]
Processed prompts:  47%|████▋     | 4682/10000 [07:02<07:51, 11.27it/s, est. speed input: 3362.76 toks/s, output: 850.57 toks/s]
Processed prompts:  47%|████▋     | 4687/10000 [07:02<06:31, 13.57it/s, est. speed input: 3363.93 toks/s, output: 850.85 toks/s]
Processed prompts:  47%|████▋     | 4689/10000 [07:02<08:01, 11.02it/s, est. speed input: 3362.51 toks/s, output: 850.50 toks/s]
Processed prompts:  47%|████▋     | 4695/10000 [07:03<05:56, 14.90it/s, est. speed input: 3364.77 toks/s, output: 851.17 toks/s]
Processed prompts:  47%|████▋     | 4699/10000 [07:03<06:47, 12.99it/s, est. speed input: 3363.85 toks/s, output: 851.02 toks/s]
Processed prompts:  47%|████▋     | 4701/10000 [07:03<07:44, 11.41it/s, est. speed input: 3362.86 toks/s, output: 850.83 toks/s]
Processed prompts:  47%|████▋     | 4705/10000 [07:04<06:28, 13.64it/s, est. speed input: 3363.84 toks/s, output: 851.19 toks/s]
Processed prompts:  47%|████▋     | 4707/10000 [07:04<08:01, 11.00it/s, est. speed input: 3362.29 toks/s, output: 850.94 toks/s]
Processed prompts:  47%|████▋     | 4710/10000 [07:04<07:46, 11.33it/s, est. speed input: 3362.11 toks/s, output: 850.94 toks/s]
Processed prompts:  47%|████▋     | 4712/10000 [07:04<08:35, 10.26it/s, est. speed input: 3361.16 toks/s, output: 850.77 toks/s]
Processed prompts:  47%|████▋     | 4716/10000 [07:05<07:18, 12.04it/s, est. speed input: 3362.00 toks/s, output: 850.99 toks/s]
Processed prompts:  47%|████▋     | 4719/10000 [07:05<07:48, 11.27it/s, est. speed input: 3361.26 toks/s, output: 850.92 toks/s]
Processed prompts:  47%|████▋     | 4721/10000 [07:05<08:31, 10.32it/s, est. speed input: 3360.61 toks/s, output: 850.77 toks/s]
Processed prompts:  47%|████▋     | 4726/10000 [07:05<06:39, 13.21it/s, est. speed input: 3361.48 toks/s, output: 851.14 toks/s]
Processed prompts:  47%|████▋     | 4728/10000 [07:06<08:18, 10.58it/s, est. speed input: 3359.84 toks/s, output: 850.91 toks/s]
Processed prompts:  47%|████▋     | 4732/10000 [07:06<07:13, 12.15it/s, est. speed input: 3360.40 toks/s, output: 851.06 toks/s]
Processed prompts:  47%|████▋     | 4734/10000 [07:06<08:18, 10.57it/s, est. speed input: 3359.27 toks/s, output: 850.79 toks/s]
Processed prompts:  47%|████▋     | 4736/10000 [07:06<08:32, 10.27it/s, est. speed input: 3358.94 toks/s, output: 850.71 toks/s]
Processed prompts:  47%|████▋     | 4738/10000 [07:07<08:17, 10.58it/s, est. speed input: 3358.96 toks/s, output: 850.74 toks/s]
Processed prompts:  47%|████▋     | 4741/10000 [07:07<07:50, 11.18it/s, est. speed input: 3358.97 toks/s, output: 850.76 toks/s]
Processed prompts:  47%|████▋     | 4744/10000 [07:07<10:29,  8.35it/s, est. speed input: 3356.22 toks/s, output: 850.16 toks/s]
Processed prompts:  48%|████▊     | 4752/10000 [07:08<06:41, 13.05it/s, est. speed input: 3358.69 toks/s, output: 850.94 toks/s]
Processed prompts:  48%|████▊     | 4756/10000 [07:08<07:11, 12.15it/s, est. speed input: 3358.10 toks/s, output: 850.88 toks/s]
Processed prompts:  48%|████▊     | 4758/10000 [07:08<08:13, 10.61it/s, est. speed input: 3357.08 toks/s, output: 850.65 toks/s]
Processed prompts:  48%|████▊     | 4760/10000 [07:09<08:25, 10.38it/s, est. speed input: 3356.60 toks/s, output: 850.61 toks/s]
Processed prompts:  48%|████▊     | 4764/10000 [07:09<07:16, 12.00it/s, est. speed input: 3357.34 toks/s, output: 850.82 toks/s]
Processed prompts:  48%|████▊     | 4772/10000 [07:09<05:17, 16.48it/s, est. speed input: 3359.68 toks/s, output: 851.63 toks/s]
Processed prompts:  48%|████▊     | 4780/10000 [07:10<05:01, 17.29it/s, est. speed input: 3361.13 toks/s, output: 852.07 toks/s]
Processed prompts:  48%|████▊     | 4782/10000 [07:10<06:50, 12.72it/s, est. speed input: 3358.91 toks/s, output: 851.57 toks/s]
Processed prompts:  48%|████▊     | 4786/10000 [07:10<06:24, 13.55it/s, est. speed input: 3359.41 toks/s, output: 851.80 toks/s]
Processed prompts:  48%|████▊     | 4791/10000 [07:11<05:55, 14.64it/s, est. speed input: 3360.11 toks/s, output: 852.19 toks/s]
Processed prompts:  48%|████▊     | 4796/10000 [07:11<05:53, 14.73it/s, est. speed input: 3360.71 toks/s, output: 852.42 toks/s]
Processed prompts:  48%|████▊     | 4798/10000 [07:11<07:09, 12.12it/s, est. speed input: 3359.31 toks/s, output: 852.12 toks/s]
Processed prompts:  48%|████▊     | 4802/10000 [07:12<06:22, 13.60it/s, est. speed input: 3359.92 toks/s, output: 852.45 toks/s]
Processed prompts:  48%|████▊     | 4808/10000 [07:12<05:28, 15.79it/s, est. speed input: 3361.50 toks/s, output: 852.96 toks/s]
Processed prompts:  48%|████▊     | 4813/10000 [07:12<05:51, 14.76it/s, est. speed input: 3361.67 toks/s, output: 853.03 toks/s]
Processed prompts:  48%|████▊     | 4818/10000 [07:13<06:02, 14.31it/s, est. speed input: 3361.85 toks/s, output: 853.20 toks/s]
Processed prompts:  48%|████▊     | 4823/10000 [07:13<07:39, 11.26it/s, est. speed input: 3359.70 toks/s, output: 852.72 toks/s]
Processed prompts:  48%|████▊     | 4832/10000 [07:14<06:22, 13.51it/s, est. speed input: 3361.23 toks/s, output: 853.26 toks/s]
Processed prompts:  48%|████▊     | 4834/10000 [07:14<08:10, 10.52it/s, est. speed input: 3358.87 toks/s, output: 852.68 toks/s]
Processed prompts:  48%|████▊     | 4839/10000 [07:14<07:09, 12.01it/s, est. speed input: 3359.33 toks/s, output: 853.01 toks/s]
Processed prompts:  48%|████▊     | 4841/10000 [07:15<08:04, 10.64it/s, est. speed input: 3358.20 toks/s, output: 852.77 toks/s]
Processed prompts:  48%|████▊     | 4845/10000 [07:15<07:12, 11.92it/s, est. speed input: 3358.74 toks/s, output: 852.99 toks/s]
Processed prompts:  49%|████▊     | 4852/10000 [07:15<05:44, 14.95it/s, est. speed input: 3360.33 toks/s, output: 853.49 toks/s]
Processed prompts:  49%|████▊     | 4854/10000 [07:16<07:40, 11.17it/s, est. speed input: 3358.07 toks/s, output: 852.99 toks/s]
Processed prompts:  49%|████▊     | 4860/10000 [07:16<06:00, 14.26it/s, est. speed input: 3359.84 toks/s, output: 853.51 toks/s]
Processed prompts:  49%|████▊     | 4863/10000 [07:16<07:05, 12.06it/s, est. speed input: 3358.73 toks/s, output: 853.25 toks/s]
Processed prompts:  49%|████▊     | 4867/10000 [07:17<08:34,  9.98it/s, est. speed input: 3356.80 toks/s, output: 852.83 toks/s]
Processed prompts:  49%|████▊     | 4872/10000 [07:17<07:58, 10.72it/s, est. speed input: 3356.58 toks/s, output: 852.92 toks/s]
Processed prompts:  49%|████▉     | 4876/10000 [07:18<06:46, 12.61it/s, est. speed input: 3357.41 toks/s, output: 853.31 toks/s]
Processed prompts:  49%|████▉     | 4878/10000 [07:18<07:41, 11.09it/s, est. speed input: 3356.34 toks/s, output: 853.05 toks/s]
Processed prompts:  49%|████▉     | 4880/10000 [07:18<08:10, 10.44it/s, est. speed input: 3355.59 toks/s, output: 852.89 toks/s]
Processed prompts:  49%|████▉     | 4883/10000 [07:18<07:34, 11.26it/s, est. speed input: 3355.56 toks/s, output: 852.94 toks/s]
Processed prompts:  49%|████▉     | 4888/10000 [07:19<06:18, 13.52it/s, est. speed input: 3356.34 toks/s, output: 853.22 toks/s]
Processed prompts:  49%|████▉     | 4893/10000 [07:19<06:05, 13.98it/s, est. speed input: 3356.82 toks/s, output: 853.45 toks/s]
Processed prompts:  49%|████▉     | 4897/10000 [07:19<06:24, 13.27it/s, est. speed input: 3356.33 toks/s, output: 853.48 toks/s]
Processed prompts:  49%|████▉     | 4902/10000 [07:20<05:51, 14.52it/s, est. speed input: 3356.90 toks/s, output: 853.74 toks/s]
Processed prompts:  49%|████▉     | 4904/10000 [07:20<07:10, 11.84it/s, est. speed input: 3355.79 toks/s, output: 853.44 toks/s]
Processed prompts:  49%|████▉     | 4909/10000 [07:20<05:49, 14.58it/s, est. speed input: 3357.22 toks/s, output: 853.84 toks/s]
Processed prompts:  49%|████▉     | 4914/10000 [07:20<05:43, 14.81it/s, est. speed input: 3357.71 toks/s, output: 853.97 toks/s]
Processed prompts:  49%|████▉     | 4917/10000 [07:21<06:50, 12.39it/s, est. speed input: 3356.68 toks/s, output: 853.79 toks/s]
Processed prompts:  49%|████▉     | 4922/10000 [07:21<06:01, 14.06it/s, est. speed input: 3357.42 toks/s, output: 854.07 toks/s]
Processed prompts:  49%|████▉     | 4927/10000 [07:21<05:50, 14.46it/s, est. speed input: 3357.89 toks/s, output: 854.22 toks/s]
Processed prompts:  49%|████▉     | 4931/10000 [07:22<06:13, 13.58it/s, est. speed input: 3357.51 toks/s, output: 854.17 toks/s]
Processed prompts:  49%|████▉     | 4939/10000 [07:22<06:52, 12.26it/s, est. speed input: 3356.16 toks/s, output: 854.04 toks/s]
Processed prompts:  49%|████▉     | 4948/10000 [07:23<05:39, 14.89it/s, est. speed input: 3358.61 toks/s, output: 854.62 toks/s]
Processed prompts:  50%|████▉     | 4950/10000 [07:23<06:29, 12.96it/s, est. speed input: 3357.49 toks/s, output: 854.38 toks/s]
Processed prompts:  50%|████▉     | 4953/10000 [07:23<06:34, 12.80it/s, est. speed input: 3357.26 toks/s, output: 854.42 toks/s]
Processed prompts:  50%|████▉     | 4955/10000 [07:24<07:14, 11.62it/s, est. speed input: 3356.51 toks/s, output: 854.28 toks/s]
Processed prompts:  50%|████▉     | 4960/10000 [07:24<05:54, 14.23it/s, est. speed input: 3357.63 toks/s, output: 854.69 toks/s]
Processed prompts:  50%|████▉     | 4962/10000 [07:24<07:14, 11.59it/s, est. speed input: 3356.40 toks/s, output: 854.45 toks/s]
Processed prompts:  50%|████▉     | 4964/10000 [07:24<07:15, 11.56it/s, est. speed input: 3356.52 toks/s, output: 854.54 toks/s]
Processed prompts:  50%|████▉     | 4967/10000 [07:25<09:19,  8.99it/s, est. speed input: 3354.34 toks/s, output: 854.14 toks/s]
Processed prompts:  50%|████▉     | 4974/10000 [07:25<06:46, 12.37it/s, est. speed input: 3356.00 toks/s, output: 854.67 toks/s]
Processed prompts:  50%|████▉     | 4976/10000 [07:26<09:33,  8.77it/s, est. speed input: 3352.89 toks/s, output: 853.91 toks/s]
Processed prompts:  50%|████▉     | 4979/10000 [07:26<09:42,  8.62it/s, est. speed input: 3351.60 toks/s, output: 853.63 toks/s]
Processed prompts:  50%|████▉     | 4983/10000 [07:26<08:36,  9.72it/s, est. speed input: 3351.41 toks/s, output: 853.67 toks/s]
Processed prompts:  50%|████▉     | 4985/10000 [07:27<08:39,  9.66it/s, est. speed input: 3350.98 toks/s, output: 853.56 toks/s]
Processed prompts:  50%|████▉     | 4987/10000 [07:27<08:43,  9.58it/s, est. speed input: 3350.59 toks/s, output: 853.49 toks/s]
Processed prompts:  50%|████▉     | 4989/10000 [07:27<08:35,  9.73it/s, est. speed input: 3350.36 toks/s, output: 853.48 toks/s]
Processed prompts:  50%|████▉     | 4993/10000 [07:27<07:20, 11.36it/s, est. speed input: 3350.57 toks/s, output: 853.70 toks/s]
Processed prompts:  50%|████▉     | 4995/10000 [07:28<08:33,  9.75it/s, est. speed input: 3349.63 toks/s, output: 853.54 toks/s]
Processed prompts:  50%|█████     | 5000/10000 [07:28<06:01, 13.82it/s, est. speed input: 3351.32 toks/s, output: 854.00 toks/s]
Processed prompts:  50%|█████     | 5005/10000 [07:28<06:09, 13.53it/s, est. speed input: 3351.43 toks/s, output: 854.09 toks/s]
Processed prompts:  50%|█████     | 5009/10000 [07:29<06:25, 12.93it/s, est. speed input: 3351.20 toks/s, output: 854.09 toks/s]
Processed prompts:  50%|█████     | 5016/10000 [07:29<05:06, 16.26it/s, est. speed input: 3353.26 toks/s, output: 854.66 toks/s]
Processed prompts:  50%|█████     | 5020/10000 [07:29<05:58, 13.90it/s, est. speed input: 3352.56 toks/s, output: 854.60 toks/s]
Processed prompts:  50%|█████     | 5023/10000 [07:30<06:21, 13.06it/s, est. speed input: 3352.10 toks/s, output: 854.58 toks/s]
Processed prompts:  50%|█████     | 5025/10000 [07:30<07:08, 11.61it/s, est. speed input: 3351.30 toks/s, output: 854.45 toks/s]
Processed prompts:  50%|█████     | 5031/10000 [07:30<05:30, 15.03it/s, est. speed input: 3353.00 toks/s, output: 855.01 toks/s]
Processed prompts:  50%|█████     | 5034/10000 [07:30<06:41, 12.37it/s, est. speed input: 3351.99 toks/s, output: 854.78 toks/s]
Processed prompts:  50%|█████     | 5039/10000 [07:31<05:43, 14.44it/s, est. speed input: 3352.96 toks/s, output: 855.14 toks/s]
Processed prompts:  50%|█████     | 5042/10000 [07:31<05:07, 16.13it/s, est. speed input: 3353.85 toks/s, output: 855.44 toks/s]
Processed prompts:  50%|█████     | 5050/10000 [07:31<03:58, 20.79it/s, est. speed input: 3356.44 toks/s, output: 856.26 toks/s]
Processed prompts:  51%|█████     | 5054/10000 [07:31<05:19, 15.48it/s, est. speed input: 3355.49 toks/s, output: 856.08 toks/s]
Processed prompts:  51%|█████     | 5057/10000 [07:32<05:50, 14.11it/s, est. speed input: 3355.12 toks/s, output: 856.01 toks/s]
Processed prompts:  51%|█████     | 5060/10000 [07:32<06:06, 13.46it/s, est. speed input: 3354.99 toks/s, output: 856.04 toks/s]
Processed prompts:  51%|█████     | 5063/10000 [07:32<06:23, 12.88it/s, est. speed input: 3354.80 toks/s, output: 856.02 toks/s]
Processed prompts:  51%|█████     | 5066/10000 [07:33<06:37, 12.43it/s, est. speed input: 3354.49 toks/s, output: 855.97 toks/s]
Processed prompts:  51%|█████     | 5070/10000 [07:33<06:13, 13.21it/s, est. speed input: 3355.03 toks/s, output: 856.07 toks/s]
Processed prompts:  51%|█████     | 5072/10000 [07:33<07:15, 11.32it/s, est. speed input: 3354.03 toks/s, output: 855.85 toks/s]
Processed prompts:  51%|█████     | 5075/10000 [07:34<09:11,  8.92it/s, est. speed input: 3351.87 toks/s, output: 855.42 toks/s]
Processed prompts:  51%|█████     | 5088/10000 [07:34<04:58, 16.48it/s, est. speed input: 3356.06 toks/s, output: 856.73 toks/s]
Processed prompts:  51%|█████     | 5090/10000 [07:34<06:38, 12.32it/s, est. speed input: 3353.99 toks/s, output: 856.22 toks/s]
Processed prompts:  51%|█████     | 5097/10000 [07:35<05:11, 15.75it/s, est. speed input: 3356.31 toks/s, output: 856.96 toks/s]
Processed prompts:  51%|█████     | 5100/10000 [07:35<06:23, 12.76it/s, est. speed input: 3354.66 toks/s, output: 856.65 toks/s]
Processed prompts:  51%|█████     | 5105/10000 [07:35<05:44, 14.20it/s, est. speed input: 3355.53 toks/s, output: 856.89 toks/s]
Processed prompts:  51%|█████     | 5107/10000 [07:36<06:55, 11.76it/s, est. speed input: 3354.24 toks/s, output: 856.61 toks/s]
Processed prompts:  51%|█████     | 5109/10000 [07:36<07:27, 10.94it/s, est. speed input: 3353.59 toks/s, output: 856.51 toks/s]
Processed prompts:  51%|█████     | 5111/10000 [07:36<07:42, 10.57it/s, est. speed input: 3353.23 toks/s, output: 856.46 toks/s]
Processed prompts:  51%|█████     | 5113/10000 [07:36<08:10,  9.96it/s, est. speed input: 3352.68 toks/s, output: 856.37 toks/s]
Processed prompts:  51%|█████     | 5117/10000 [07:37<06:37, 12.27it/s, est. speed input: 3353.34 toks/s, output: 856.67 toks/s]
Processed prompts:  51%|█████     | 5124/10000 [07:37<05:01, 16.20it/s, est. speed input: 3355.10 toks/s, output: 857.30 toks/s]
Processed prompts:  51%|█████▏    | 5128/10000 [07:37<06:05, 13.31it/s, est. speed input: 3354.29 toks/s, output: 857.15 toks/s]
Processed prompts:  51%|█████▏    | 5130/10000 [07:38<07:13, 11.23it/s, est. speed input: 3353.05 toks/s, output: 856.87 toks/s]
Processed prompts:  51%|█████▏    | 5132/10000 [07:38<07:10, 11.32it/s, est. speed input: 3353.01 toks/s, output: 856.90 toks/s]
Processed prompts:  51%|█████▏    | 5135/10000 [07:38<07:02, 11.52it/s, est. speed input: 3352.85 toks/s, output: 856.92 toks/s]
Processed prompts:  51%|█████▏    | 5137/10000 [07:38<07:58, 10.15it/s, est. speed input: 3351.95 toks/s, output: 856.75 toks/s]
Processed prompts:  51%|█████▏    | 5142/10000 [07:39<06:11, 13.06it/s, est. speed input: 3353.06 toks/s, output: 857.15 toks/s]
Processed prompts:  51%|█████▏    | 5145/10000 [07:39<06:57, 11.62it/s, est. speed input: 3352.19 toks/s, output: 856.95 toks/s]
Processed prompts:  51%|█████▏    | 5147/10000 [07:39<07:45, 10.43it/s, est. speed input: 3351.31 toks/s, output: 856.80 toks/s]
Processed prompts:  51%|█████▏    | 5149/10000 [07:39<08:12,  9.86it/s, est. speed input: 3350.73 toks/s, output: 856.70 toks/s]
Processed prompts:  52%|█████▏    | 5152/10000 [07:40<07:23, 10.94it/s, est. speed input: 3351.01 toks/s, output: 856.78 toks/s]
Processed prompts:  52%|█████▏    | 5155/10000 [07:40<07:18, 11.04it/s, est. speed input: 3350.78 toks/s, output: 856.72 toks/s]
Processed prompts:  52%|█████▏    | 5161/10000 [07:40<05:33, 14.52it/s, est. speed input: 3352.13 toks/s, output: 857.17 toks/s]
Processed prompts:  52%|█████▏    | 5167/10000 [07:41<05:24, 14.90it/s, est. speed input: 3352.73 toks/s, output: 857.39 toks/s]
Processed prompts:  52%|█████▏    | 5172/10000 [07:41<05:23, 14.92it/s, est. speed input: 3353.01 toks/s, output: 857.51 toks/s]
Processed prompts:  52%|█████▏    | 5175/10000 [07:41<06:08, 13.11it/s, est. speed input: 3352.32 toks/s, output: 857.37 toks/s]
Processed prompts:  52%|█████▏    | 5177/10000 [07:42<06:54, 11.64it/s, est. speed input: 3351.46 toks/s, output: 857.21 toks/s]
Processed prompts:  52%|█████▏    | 5183/10000 [07:42<05:10, 15.53it/s, est. speed input: 3353.28 toks/s, output: 857.81 toks/s]
Processed prompts:  52%|█████▏    | 5190/10000 [07:42<04:57, 16.16it/s, est. speed input: 3354.50 toks/s, output: 858.26 toks/s]
Processed prompts:  52%|█████▏    | 5193/10000 [07:43<06:08, 13.04it/s, est. speed input: 3353.34 toks/s, output: 858.00 toks/s]
Processed prompts:  52%|█████▏    | 5195/10000 [07:43<06:51, 11.69it/s, est. speed input: 3352.50 toks/s, output: 857.87 toks/s]
Processed prompts:  52%|█████▏    | 5197/10000 [07:43<06:59, 11.45it/s, est. speed input: 3352.19 toks/s, output: 857.83 toks/s]
Processed prompts:  52%|█████▏    | 5199/10000 [07:43<07:45, 10.32it/s, est. speed input: 3351.53 toks/s, output: 857.68 toks/s]
Processed prompts:  52%|█████▏    | 5201/10000 [07:43<07:45, 10.31it/s, est. speed input: 3351.24 toks/s, output: 857.63 toks/s]
Processed prompts:  52%|█████▏    | 5206/10000 [07:44<06:04, 13.15it/s, est. speed input: 3352.16 toks/s, output: 857.88 toks/s]
Processed prompts:  52%|█████▏    | 5209/10000 [07:44<06:48, 11.73it/s, est. speed input: 3351.41 toks/s, output: 857.82 toks/s]
Processed prompts:  52%|█████▏    | 5213/10000 [07:44<06:12, 12.87it/s, est. speed input: 3351.78 toks/s, output: 858.04 toks/s]
Processed prompts:  52%|█████▏    | 5222/10000 [07:45<04:20, 18.34it/s, est. speed input: 3354.28 toks/s, output: 858.88 toks/s]
Processed prompts:  52%|█████▏    | 5226/10000 [07:45<05:34, 14.28it/s, est. speed input: 3353.07 toks/s, output: 858.67 toks/s]
Processed prompts:  52%|█████▏    | 5228/10000 [07:45<06:26, 12.36it/s, est. speed input: 3352.08 toks/s, output: 858.44 toks/s]
Processed prompts:  52%|█████▏    | 5232/10000 [07:46<05:45, 13.80it/s, est. speed input: 3352.76 toks/s, output: 858.62 toks/s]
Processed prompts:  52%|█████▏    | 5235/10000 [07:46<07:54, 10.05it/s, est. speed input: 3350.49 toks/s, output: 858.08 toks/s]
Processed prompts:  52%|█████▏    | 5242/10000 [07:46<06:10, 12.84it/s, est. speed input: 3351.91 toks/s, output: 858.47 toks/s]
Processed prompts:  52%|█████▏    | 5249/10000 [07:47<06:49, 11.59it/s, est. speed input: 3350.73 toks/s, output: 858.25 toks/s]
Processed prompts:  53%|█████▎    | 5256/10000 [07:47<05:35, 14.16it/s, est. speed input: 3352.12 toks/s, output: 858.78 toks/s]
Processed prompts:  53%|█████▎    | 5259/10000 [07:48<06:10, 12.81it/s, est. speed input: 3351.24 toks/s, output: 858.61 toks/s]
Processed prompts:  53%|█████▎    | 5264/10000 [07:48<05:33, 14.19it/s, est. speed input: 3351.97 toks/s, output: 858.81 toks/s]
Processed prompts:  53%|█████▎    | 5266/10000 [07:48<06:40, 11.81it/s, est. speed input: 3350.70 toks/s, output: 858.49 toks/s]
Processed prompts:  53%|█████▎    | 5269/10000 [07:49<06:24, 12.32it/s, est. speed input: 3350.92 toks/s, output: 858.55 toks/s]
Processed prompts:  53%|█████▎    | 5273/10000 [07:49<05:57, 13.21it/s, est. speed input: 3351.19 toks/s, output: 858.64 toks/s]
Processed prompts:  53%|█████▎    | 5277/10000 [07:49<06:03, 12.99it/s, est. speed input: 3351.25 toks/s, output: 858.59 toks/s]
Processed prompts:  53%|█████▎    | 5279/10000 [07:49<06:59, 11.26it/s, est. speed input: 3350.39 toks/s, output: 858.40 toks/s]
Processed prompts:  53%|█████▎    | 5282/10000 [07:50<06:34, 11.95it/s, est. speed input: 3350.59 toks/s, output: 858.49 toks/s]
Processed prompts:  53%|█████▎    | 5285/10000 [07:50<08:40,  9.06it/s, est. speed input: 3348.43 toks/s, output: 857.99 toks/s]
Processed prompts:  53%|█████▎    | 5289/10000 [07:51<07:57,  9.87it/s, est. speed input: 3347.94 toks/s, output: 857.93 toks/s]
Processed prompts:  53%|█████▎    | 5293/10000 [07:51<06:51, 11.43it/s, est. speed input: 3348.46 toks/s, output: 858.03 toks/s]
Processed prompts:  53%|█████▎    | 5295/10000 [07:51<07:52,  9.95it/s, est. speed input: 3347.37 toks/s, output: 857.76 toks/s]
Processed prompts:  53%|█████▎    | 5298/10000 [07:51<07:09, 10.95it/s, est. speed input: 3347.51 toks/s, output: 857.87 toks/s]
Processed prompts:  53%|█████▎    | 5303/10000 [07:52<05:50, 13.39it/s, est. speed input: 3348.30 toks/s, output: 858.28 toks/s]
Processed prompts:  53%|█████▎    | 5305/10000 [07:52<07:15, 10.79it/s, est. speed input: 3347.00 toks/s, output: 858.03 toks/s]
Processed prompts:  53%|█████▎    | 5308/10000 [07:52<06:34, 11.88it/s, est. speed input: 3347.20 toks/s, output: 858.20 toks/s]
Processed prompts:  53%|█████▎    | 5312/10000 [07:52<06:15, 12.49it/s, est. speed input: 3347.31 toks/s, output: 858.22 toks/s]
Processed prompts:  53%|█████▎    | 5314/10000 [07:53<07:13, 10.81it/s, est. speed input: 3346.45 toks/s, output: 858.01 toks/s]
Processed prompts:  53%|█████▎    | 5320/10000 [07:53<05:08, 15.17it/s, est. speed input: 3348.27 toks/s, output: 858.51 toks/s]
Processed prompts:  53%|█████▎    | 5325/10000 [07:53<05:23, 14.47it/s, est. speed input: 3348.62 toks/s, output: 858.56 toks/s]
Processed prompts:  53%|█████▎    | 5328/10000 [07:54<06:09, 12.66it/s, est. speed input: 3347.77 toks/s, output: 858.46 toks/s]
Processed prompts:  53%|█████▎    | 5333/10000 [07:54<05:21, 14.54it/s, est. speed input: 3348.84 toks/s, output: 858.84 toks/s]
Processed prompts:  53%|█████▎    | 5341/10000 [07:54<04:21, 17.85it/s, est. speed input: 3351.18 toks/s, output: 859.51 toks/s]
Processed prompts:  53%|█████▎    | 5345/10000 [07:55<05:20, 14.54it/s, est. speed input: 3350.13 toks/s, output: 859.32 toks/s]
Processed prompts:  53%|█████▎    | 5347/10000 [07:55<06:18, 12.30it/s, est. speed input: 3349.21 toks/s, output: 859.05 toks/s]
Processed prompts:  53%|█████▎    | 5349/10000 [07:55<06:37, 11.70it/s, est. speed input: 3348.79 toks/s, output: 859.00 toks/s]
Processed prompts:  54%|█████▎    | 5354/10000 [07:55<05:19, 14.55it/s, est. speed input: 3350.09 toks/s, output: 859.46 toks/s]
Processed prompts:  54%|█████▎    | 5356/10000 [07:56<06:41, 11.56it/s, est. speed input: 3348.85 toks/s, output: 859.20 toks/s]
Processed prompts:  54%|█████▎    | 5359/10000 [07:56<06:20, 12.20it/s, est. speed input: 3348.84 toks/s, output: 859.28 toks/s]
Processed prompts:  54%|█████▎    | 5366/10000 [07:56<04:42, 16.43it/s, est. speed input: 3350.64 toks/s, output: 859.82 toks/s]
Processed prompts:  54%|█████▎    | 5368/10000 [07:57<08:05,  9.53it/s, est. speed input: 3347.23 toks/s, output: 859.00 toks/s]
Processed prompts:  54%|█████▍    | 5376/10000 [07:57<04:59, 15.46it/s, est. speed input: 3350.38 toks/s, output: 859.95 toks/s]
Processed prompts:  54%|█████▍    | 5380/10000 [07:58<07:31, 10.24it/s, est. speed input: 3347.34 toks/s, output: 859.22 toks/s]
Processed prompts:  54%|█████▍    | 5385/10000 [07:58<07:03, 10.89it/s, est. speed input: 3347.44 toks/s, output: 859.22 toks/s]
Processed prompts:  54%|█████▍    | 5388/10000 [07:58<06:35, 11.68it/s, est. speed input: 3347.64 toks/s, output: 859.36 toks/s]
Processed prompts:  54%|█████▍    | 5390/10000 [07:59<07:10, 10.70it/s, est. speed input: 3346.94 toks/s, output: 859.26 toks/s]
Processed prompts:  54%|█████▍    | 5393/10000 [07:59<06:41, 11.48it/s, est. speed input: 3347.32 toks/s, output: 859.40 toks/s]
Processed prompts:  54%|█████▍    | 5396/10000 [07:59<06:47, 11.31it/s, est. speed input: 3347.02 toks/s, output: 859.36 toks/s]
Processed prompts:  54%|█████▍    | 5399/10000 [07:59<06:48, 11.26it/s, est. speed input: 3346.68 toks/s, output: 859.35 toks/s]
Processed prompts:  54%|█████▍    | 5402/10000 [08:00<06:42, 11.44it/s, est. speed input: 3346.54 toks/s, output: 859.38 toks/s]
Processed prompts:  54%|█████▍    | 5406/10000 [08:00<06:06, 12.55it/s, est. speed input: 3346.94 toks/s, output: 859.56 toks/s]
Processed prompts:  54%|█████▍    | 5408/10000 [08:00<07:11, 10.64it/s, est. speed input: 3345.73 toks/s, output: 859.32 toks/s]
Processed prompts:  54%|█████▍    | 5410/10000 [08:00<07:23, 10.35it/s, est. speed input: 3345.16 toks/s, output: 859.25 toks/s]
Processed prompts:  54%|█████▍    | 5416/10000 [08:01<05:18, 14.39it/s, est. speed input: 3346.30 toks/s, output: 859.75 toks/s]
Processed prompts:  54%|█████▍    | 5418/10000 [08:01<08:28,  9.02it/s, est. speed input: 3343.52 toks/s, output: 859.07 toks/s]
Processed prompts:  54%|█████▍    | 5422/10000 [08:01<06:46, 11.28it/s, est. speed input: 3344.20 toks/s, output: 859.33 toks/s]
Processed prompts:  54%|█████▍    | 5424/10000 [08:02<07:50,  9.73it/s, est. speed input: 3342.99 toks/s, output: 859.10 toks/s]
Processed prompts:  54%|█████▍    | 5430/10000 [08:02<05:26, 13.98it/s, est. speed input: 3344.52 toks/s, output: 859.66 toks/s]
Processed prompts:  54%|█████▍    | 5436/10000 [08:02<05:10, 14.70it/s, est. speed input: 3345.15 toks/s, output: 859.92 toks/s]
Processed prompts:  54%|█████▍    | 5444/10000 [08:03<04:30, 16.86it/s, est. speed input: 3346.28 toks/s, output: 860.44 toks/s]
Processed prompts:  55%|█████▍    | 5450/10000 [08:03<04:48, 15.75it/s, est. speed input: 3346.95 toks/s, output: 860.65 toks/s]
Processed prompts:  55%|█████▍    | 5457/10000 [08:03<04:34, 16.52it/s, est. speed input: 3348.04 toks/s, output: 861.05 toks/s]
Processed prompts:  55%|█████▍    | 5461/10000 [08:04<05:15, 14.38it/s, est. speed input: 3347.53 toks/s, output: 860.91 toks/s]
Processed prompts:  55%|█████▍    | 5465/10000 [08:04<05:16, 14.32it/s, est. speed input: 3347.94 toks/s, output: 861.00 toks/s]
Processed prompts:  55%|█████▍    | 5467/10000 [08:04<06:12, 12.17it/s, est. speed input: 3347.15 toks/s, output: 860.80 toks/s]
Processed prompts:  55%|█████▍    | 5472/10000 [08:05<05:09, 14.62it/s, est. speed input: 3348.78 toks/s, output: 861.15 toks/s]
Processed prompts:  55%|█████▍    | 5476/10000 [08:05<05:28, 13.78it/s, est. speed input: 3348.83 toks/s, output: 861.18 toks/s]
Processed prompts:  55%|█████▍    | 5478/10000 [08:05<06:21, 11.85it/s, est. speed input: 3347.74 toks/s, output: 860.95 toks/s]
Processed prompts:  55%|█████▍    | 5480/10000 [08:06<07:56,  9.49it/s, est. speed input: 3346.21 toks/s, output: 860.58 toks/s]
Processed prompts:  55%|█████▍    | 5483/10000 [08:06<07:00, 10.74it/s, est. speed input: 3346.41 toks/s, output: 860.68 toks/s]
Processed prompts:  55%|█████▍    | 5487/10000 [08:06<06:23, 11.78it/s, est. speed input: 3346.48 toks/s, output: 860.71 toks/s]
Processed prompts:  55%|█████▍    | 5489/10000 [08:07<08:45,  8.59it/s, est. speed input: 3344.21 toks/s, output: 860.17 toks/s]
Processed prompts:  55%|█████▍    | 5495/10000 [08:07<07:10, 10.46it/s, est. speed input: 3344.43 toks/s, output: 860.40 toks/s]
Processed prompts:  55%|█████▌    | 5500/10000 [08:07<05:44, 13.06it/s, est. speed input: 3345.39 toks/s, output: 860.80 toks/s]
Processed prompts:  55%|█████▌    | 5502/10000 [08:08<06:55, 10.82it/s, est. speed input: 3344.15 toks/s, output: 860.53 toks/s]
Processed prompts:  55%|█████▌    | 5510/10000 [08:08<04:38, 16.13it/s, est. speed input: 3346.99 toks/s, output: 861.38 toks/s]
Processed prompts:  55%|█████▌    | 5512/10000 [08:08<06:24, 11.69it/s, est. speed input: 3345.18 toks/s, output: 860.94 toks/s]
Processed prompts:  55%|█████▌    | 5514/10000 [08:09<06:50, 10.93it/s, est. speed input: 3344.48 toks/s, output: 860.81 toks/s]
Processed prompts:  55%|█████▌    | 5516/10000 [08:09<07:02, 10.61it/s, est. speed input: 3344.28 toks/s, output: 860.85 toks/s]
Processed prompts:  55%|█████▌    | 5523/10000 [08:09<04:26, 16.80it/s, est. speed input: 3346.49 toks/s, output: 861.56 toks/s]
Processed prompts:  55%|█████▌    | 5529/10000 [08:09<04:46, 15.61it/s, est. speed input: 3346.37 toks/s, output: 861.62 toks/s]
Processed prompts:  55%|█████▌    | 5531/10000 [08:10<05:59, 12.44it/s, est. speed input: 3345.14 toks/s, output: 861.35 toks/s]
Processed prompts:  55%|█████▌    | 5533/10000 [08:10<06:04, 12.26it/s, est. speed input: 3344.99 toks/s, output: 861.32 toks/s]
Processed prompts:  55%|█████▌    | 5535/10000 [08:10<06:26, 11.55it/s, est. speed input: 3344.69 toks/s, output: 861.28 toks/s]
Processed prompts:  55%|█████▌    | 5538/10000 [08:10<06:03, 12.26it/s, est. speed input: 3344.76 toks/s, output: 861.39 toks/s]
Processed prompts:  55%|█████▌    | 5546/10000 [08:11<04:11, 17.73it/s, est. speed input: 3347.13 toks/s, output: 862.10 toks/s]
Processed prompts:  55%|█████▌    | 5549/10000 [08:11<05:37, 13.18it/s, est. speed input: 3346.02 toks/s, output: 861.82 toks/s]
Processed prompts:  56%|█████▌    | 5554/10000 [08:11<04:58, 14.88it/s, est. speed input: 3346.76 toks/s, output: 862.12 toks/s]
Processed prompts:  56%|█████▌    | 5557/10000 [08:12<05:39, 13.09it/s, est. speed input: 3346.05 toks/s, output: 862.03 toks/s]
Processed prompts:  56%|█████▌    | 5559/10000 [08:12<06:17, 11.75it/s, est. speed input: 3345.44 toks/s, output: 861.88 toks/s]
Processed prompts:  56%|█████▌    | 5562/10000 [08:12<05:59, 12.36it/s, est. speed input: 3345.57 toks/s, output: 861.95 toks/s]
Processed prompts:  56%|█████▌    | 5565/10000 [08:12<06:05, 12.13it/s, est. speed input: 3345.31 toks/s, output: 861.93 toks/s]
Processed prompts:  56%|█████▌    | 5571/10000 [08:13<04:44, 15.56it/s, est. speed input: 3346.55 toks/s, output: 862.39 toks/s]
Processed prompts:  56%|█████▌    | 5575/10000 [08:13<05:22, 13.71it/s, est. speed input: 3346.43 toks/s, output: 862.38 toks/s]
Processed prompts:  56%|█████▌    | 5583/10000 [08:13<04:33, 16.17it/s, est. speed input: 3348.02 toks/s, output: 862.92 toks/s]
Processed prompts:  56%|█████▌    | 5585/10000 [08:14<04:59, 14.72it/s, est. speed input: 3347.68 toks/s, output: 862.83 toks/s]
Processed prompts:  56%|█████▌    | 5587/10000 [08:14<05:12, 14.13it/s, est. speed input: 3347.54 toks/s, output: 862.88 toks/s]
Processed prompts:  56%|█████▌    | 5589/10000 [08:14<05:43, 12.83it/s, est. speed input: 3347.03 toks/s, output: 862.84 toks/s]
Processed prompts:  56%|█████▌    | 5594/10000 [08:14<04:54, 14.97it/s, est. speed input: 3347.80 toks/s, output: 863.17 toks/s]
Processed prompts:  56%|█████▌    | 5598/10000 [08:14<05:12, 14.08it/s, est. speed input: 3347.52 toks/s, output: 863.19 toks/s]
Processed prompts:  56%|█████▌    | 5600/10000 [08:15<07:16, 10.08it/s, est. speed input: 3345.74 toks/s, output: 862.74 toks/s]
Processed prompts:  56%|█████▌    | 5606/10000 [08:15<06:17, 11.63it/s, est. speed input: 3346.12 toks/s, output: 862.90 toks/s]
Processed prompts:  56%|█████▌    | 5609/10000 [08:16<06:00, 12.19it/s, est. speed input: 3346.25 toks/s, output: 862.95 toks/s]
Processed prompts:  56%|█████▌    | 5611/10000 [08:16<06:37, 11.03it/s, est. speed input: 3345.64 toks/s, output: 862.80 toks/s]
Processed prompts:  56%|█████▌    | 5615/10000 [08:16<05:39, 12.92it/s, est. speed input: 3346.47 toks/s, output: 863.05 toks/s]
Processed prompts:  56%|█████▌    | 5620/10000 [08:16<05:02, 14.50it/s, est. speed input: 3347.23 toks/s, output: 863.28 toks/s]
Processed prompts:  56%|█████▋    | 5626/10000 [08:17<04:31, 16.10it/s, est. speed input: 3348.53 toks/s, output: 863.66 toks/s]
Processed prompts:  56%|█████▋    | 5628/10000 [08:17<06:45, 10.77it/s, est. speed input: 3346.11 toks/s, output: 863.11 toks/s]
Processed prompts:  56%|█████▋    | 5631/10000 [08:17<06:07, 11.87it/s, est. speed input: 3346.70 toks/s, output: 863.30 toks/s]
Processed prompts:  56%|█████▋    | 5633/10000 [08:18<06:43, 10.82it/s, est. speed input: 3346.06 toks/s, output: 863.21 toks/s]
Processed prompts:  56%|█████▋    | 5636/10000 [08:18<06:13, 11.67it/s, est. speed input: 3346.29 toks/s, output: 863.33 toks/s]
Processed prompts:  56%|█████▋    | 5642/10000 [08:18<04:47, 15.17it/s, est. speed input: 3347.93 toks/s, output: 863.71 toks/s]
Processed prompts:  56%|█████▋    | 5645/10000 [08:18<05:35, 12.97it/s, est. speed input: 3347.29 toks/s, output: 863.58 toks/s]
Processed prompts:  56%|█████▋    | 5649/10000 [08:19<05:17, 13.68it/s, est. speed input: 3347.61 toks/s, output: 863.74 toks/s]
Processed prompts:  57%|█████▋    | 5653/10000 [08:19<05:13, 13.87it/s, est. speed input: 3347.80 toks/s, output: 863.94 toks/s]
Processed prompts:  57%|█████▋    | 5657/10000 [08:19<05:10, 13.97it/s, est. speed input: 3347.93 toks/s, output: 864.09 toks/s]
Processed prompts:  57%|█████▋    | 5660/10000 [08:19<05:33, 13.01it/s, est. speed input: 3347.48 toks/s, output: 864.04 toks/s]
Processed prompts:  57%|█████▋    | 5664/10000 [08:20<05:15, 13.76it/s, est. speed input: 3347.91 toks/s, output: 864.16 toks/s]
Processed prompts:  57%|█████▋    | 5668/10000 [08:20<05:11, 13.91it/s, est. speed input: 3348.16 toks/s, output: 864.17 toks/s]
Processed prompts:  57%|█████▋    | 5670/10000 [08:20<06:06, 11.83it/s, est. speed input: 3347.30 toks/s, output: 863.97 toks/s]
Processed prompts:  57%|█████▋    | 5673/10000 [08:20<05:32, 13.01it/s, est. speed input: 3347.62 toks/s, output: 864.08 toks/s]
Processed prompts:  57%|█████▋    | 5676/10000 [08:21<05:48, 12.42it/s, est. speed input: 3347.34 toks/s, output: 864.04 toks/s]
Processed prompts:  57%|█████▋    | 5678/10000 [08:21<06:30, 11.07it/s, est. speed input: 3346.91 toks/s, output: 863.90 toks/s]
Processed prompts:  57%|█████▋    | 5680/10000 [08:21<06:27, 11.16it/s, est. speed input: 3346.94 toks/s, output: 863.95 toks/s]
Processed prompts:  57%|█████▋    | 5683/10000 [08:21<05:59, 12.01it/s, est. speed input: 3347.27 toks/s, output: 864.05 toks/s]
Processed prompts:  57%|█████▋    | 5692/10000 [08:22<03:48, 18.88it/s, est. speed input: 3349.97 toks/s, output: 864.86 toks/s]
Processed prompts:  57%|█████▋    | 5696/10000 [08:22<04:56, 14.51it/s, est. speed input: 3348.79 toks/s, output: 864.65 toks/s]
Processed prompts:  57%|█████▋    | 5698/10000 [08:22<05:44, 12.50it/s, est. speed input: 3347.93 toks/s, output: 864.48 toks/s]
Processed prompts:  57%|█████▋    | 5701/10000 [08:23<05:32, 12.95it/s, est. speed input: 3348.26 toks/s, output: 864.54 toks/s]
Processed prompts:  57%|█████▋    | 5705/10000 [08:23<05:10, 13.82it/s, est. speed input: 3348.79 toks/s, output: 864.67 toks/s]
Processed prompts:  57%|█████▋    | 5707/10000 [08:23<05:58, 11.98it/s, est. speed input: 3348.06 toks/s, output: 864.51 toks/s]
Processed prompts:  57%|█████▋    | 5709/10000 [08:23<06:16, 11.41it/s, est. speed input: 3347.81 toks/s, output: 864.44 toks/s]
Processed prompts:  57%|█████▋    | 5713/10000 [08:23<05:17, 13.51it/s, est. speed input: 3348.51 toks/s, output: 864.67 toks/s]
Processed prompts:  57%|█████▋    | 5718/10000 [08:24<04:47, 14.89it/s, est. speed input: 3349.21 toks/s, output: 864.92 toks/s]
Processed prompts:  57%|█████▋    | 5722/10000 [08:24<05:05, 14.02it/s, est. speed input: 3348.84 toks/s, output: 864.91 toks/s]
Processed prompts:  57%|█████▋    | 5726/10000 [08:24<05:02, 14.11it/s, est. speed input: 3348.97 toks/s, output: 865.09 toks/s]
Processed prompts:  57%|█████▋    | 5731/10000 [08:25<04:38, 15.32it/s, est. speed input: 3349.82 toks/s, output: 865.42 toks/s]
Processed prompts:  57%|█████▋    | 5734/10000 [08:25<05:18, 13.41it/s, est. speed input: 3349.24 toks/s, output: 865.38 toks/s]
Processed prompts:  57%|█████▋    | 5736/10000 [08:25<05:57, 11.93it/s, est. speed input: 3348.59 toks/s, output: 865.28 toks/s]
Processed prompts:  57%|█████▋    | 5741/10000 [08:26<06:32, 10.86it/s, est. speed input: 3348.02 toks/s, output: 865.15 toks/s]
Processed prompts:  57%|█████▋    | 5749/10000 [08:26<04:41, 15.10it/s, est. speed input: 3349.95 toks/s, output: 865.88 toks/s]
Processed prompts:  58%|█████▊    | 5753/10000 [08:26<04:59, 14.18it/s, est. speed input: 3349.80 toks/s, output: 865.89 toks/s]
Processed prompts:  58%|█████▊    | 5756/10000 [08:27<05:20, 13.25it/s, est. speed input: 3349.54 toks/s, output: 865.85 toks/s]
Processed prompts:  58%|█████▊    | 5758/10000 [08:27<05:57, 11.87it/s, est. speed input: 3348.80 toks/s, output: 865.70 toks/s]
Processed prompts:  58%|█████▊    | 5760/10000 [08:27<06:12, 11.40it/s, est. speed input: 3348.51 toks/s, output: 865.68 toks/s]
Processed prompts:  58%|█████▊    | 5762/10000 [08:27<06:25, 10.99it/s, est. speed input: 3348.13 toks/s, output: 865.61 toks/s]
Processed prompts:  58%|█████▊    | 5766/10000 [08:28<05:20, 13.23it/s, est. speed input: 3348.54 toks/s, output: 865.86 toks/s]
Processed prompts:  58%|█████▊    | 5768/10000 [08:28<06:16, 11.24it/s, est. speed input: 3347.69 toks/s, output: 865.70 toks/s]
Processed prompts:  58%|█████▊    | 5771/10000 [08:28<05:51, 12.03it/s, est. speed input: 3347.81 toks/s, output: 865.79 toks/s]
Processed prompts:  58%|█████▊    | 5773/10000 [08:28<06:31, 10.80it/s, est. speed input: 3347.10 toks/s, output: 865.64 toks/s]
Processed prompts:  58%|█████▊    | 5775/10000 [08:28<06:46, 10.38it/s, est. speed input: 3346.76 toks/s, output: 865.57 toks/s]
Processed prompts:  58%|█████▊    | 5780/10000 [08:29<04:59, 14.10it/s, est. speed input: 3347.92 toks/s, output: 866.02 toks/s]
Processed prompts:  58%|█████▊    | 5782/10000 [08:29<06:16, 11.20it/s, est. speed input: 3346.80 toks/s, output: 865.75 toks/s]
Processed prompts:  58%|█████▊    | 5784/10000 [08:29<08:15,  8.50it/s, est. speed input: 3344.91 toks/s, output: 865.32 toks/s]
Processed prompts:  58%|█████▊    | 5788/10000 [08:30<08:38,  8.13it/s, est. speed input: 3343.34 toks/s, output: 865.06 toks/s]
Processed prompts:  58%|█████▊    | 5800/10000 [08:30<04:55, 14.22it/s, est. speed input: 3345.84 toks/s, output: 866.11 toks/s]
Processed prompts:  58%|█████▊    | 5803/10000 [08:31<05:22, 13.01it/s, est. speed input: 3345.19 toks/s, output: 865.95 toks/s]
Processed prompts:  58%|█████▊    | 5809/10000 [08:31<04:32, 15.40it/s, est. speed input: 3346.36 toks/s, output: 866.51 toks/s]
Processed prompts:  58%|█████▊    | 5812/10000 [08:31<05:11, 13.44it/s, est. speed input: 3345.52 toks/s, output: 866.40 toks/s]
Processed prompts:  58%|█████▊    | 5816/10000 [08:32<04:58, 14.01it/s, est. speed input: 3345.68 toks/s, output: 866.52 toks/s]
Processed prompts:  58%|█████▊    | 5819/10000 [08:32<05:19, 13.09it/s, est. speed input: 3345.20 toks/s, output: 866.50 toks/s]
Processed prompts:  58%|█████▊    | 5824/10000 [08:32<06:13, 11.17it/s, est. speed input: 3343.77 toks/s, output: 866.32 toks/s]
Processed prompts:  58%|█████▊    | 5832/10000 [08:33<05:00, 13.89it/s, est. speed input: 3344.81 toks/s, output: 866.88 toks/s]
Processed prompts:  58%|█████▊    | 5834/10000 [08:33<05:33, 12.50it/s, est. speed input: 3344.04 toks/s, output: 866.77 toks/s]
Processed prompts:  58%|█████▊    | 5836/10000 [08:33<05:33, 12.47it/s, est. speed input: 3343.91 toks/s, output: 866.79 toks/s]
Processed prompts:  58%|█████▊    | 5840/10000 [08:34<05:12, 13.31it/s, est. speed input: 3344.02 toks/s, output: 866.95 toks/s]
Processed prompts:  58%|█████▊    | 5844/10000 [08:34<05:04, 13.63it/s, est. speed input: 3344.03 toks/s, output: 867.12 toks/s]
Processed prompts:  58%|█████▊    | 5846/10000 [08:34<05:57, 11.61it/s, est. speed input: 3343.06 toks/s, output: 866.95 toks/s]
Processed prompts:  58%|█████▊    | 5849/10000 [08:34<05:39, 12.22it/s, est. speed input: 3343.06 toks/s, output: 867.07 toks/s]
Processed prompts:  59%|█████▊    | 5853/10000 [08:35<05:12, 13.27it/s, est. speed input: 3343.15 toks/s, output: 867.25 toks/s]
Processed prompts:  59%|█████▊    | 5860/10000 [08:35<04:03, 16.97it/s, est. speed input: 3344.48 toks/s, output: 867.83 toks/s]
Processed prompts:  59%|█████▊    | 5864/10000 [08:35<04:46, 14.42it/s, est. speed input: 3343.74 toks/s, output: 867.80 toks/s]
Processed prompts:  59%|█████▊    | 5868/10000 [08:35<04:46, 14.44it/s, est. speed input: 3343.81 toks/s, output: 867.98 toks/s]
Processed prompts:  59%|█████▊    | 5873/10000 [08:36<04:24, 15.63it/s, est. speed input: 3344.31 toks/s, output: 868.23 toks/s]
Processed prompts:  59%|█████▉    | 5875/10000 [08:36<06:19, 10.87it/s, est. speed input: 3342.11 toks/s, output: 867.75 toks/s]
Processed prompts:  59%|█████▉    | 5877/10000 [08:36<06:14, 11.02it/s, est. speed input: 3341.91 toks/s, output: 867.72 toks/s]
Processed prompts:  59%|█████▉    | 5879/10000 [08:37<06:26, 10.68it/s, est. speed input: 3341.45 toks/s, output: 867.67 toks/s]
Processed prompts:  59%|█████▉    | 5882/10000 [08:37<05:39, 12.13it/s, est. speed input: 3341.68 toks/s, output: 867.79 toks/s]
Processed prompts:  59%|█████▉    | 5887/10000 [08:37<04:56, 13.89it/s, est. speed input: 3342.03 toks/s, output: 868.00 toks/s]
Processed prompts:  59%|█████▉    | 5891/10000 [08:37<05:01, 13.62it/s, est. speed input: 3341.83 toks/s, output: 868.01 toks/s]
Processed prompts:  59%|█████▉    | 5894/10000 [08:38<05:22, 12.72it/s, est. speed input: 3341.36 toks/s, output: 868.02 toks/s]
Processed prompts:  59%|█████▉    | 5897/10000 [08:38<05:28, 12.49it/s, est. speed input: 3341.06 toks/s, output: 868.07 toks/s]
Processed prompts:  59%|█████▉    | 5900/10000 [08:38<05:33, 12.30it/s, est. speed input: 3340.77 toks/s, output: 868.12 toks/s]
Processed prompts:  59%|█████▉    | 5902/10000 [08:38<06:11, 11.02it/s, est. speed input: 3340.01 toks/s, output: 867.98 toks/s]
Processed prompts:  59%|█████▉    | 5908/10000 [08:39<04:22, 15.59it/s, est. speed input: 3341.38 toks/s, output: 868.43 toks/s]
Processed prompts:  59%|█████▉    | 5915/10000 [08:39<03:52, 17.53it/s, est. speed input: 3342.34 toks/s, output: 868.92 toks/s]
Processed prompts:  59%|█████▉    | 5919/10000 [08:39<04:35, 14.83it/s, est. speed input: 3341.64 toks/s, output: 868.94 toks/s]
Processed prompts:  59%|█████▉    | 5923/10000 [08:40<04:36, 14.73it/s, est. speed input: 3341.62 toks/s, output: 869.11 toks/s]
Processed prompts:  59%|█████▉    | 5926/10000 [08:40<04:59, 13.61it/s, est. speed input: 3341.17 toks/s, output: 869.10 toks/s]
Processed prompts:  59%|█████▉    | 5930/10000 [08:40<04:47, 14.17it/s, est. speed input: 3341.31 toks/s, output: 869.25 toks/s]
Processed prompts:  59%|█████▉    | 5934/10000 [08:40<04:45, 14.26it/s, est. speed input: 3341.32 toks/s, output: 869.40 toks/s]
Processed prompts:  59%|█████▉    | 5938/10000 [08:41<04:43, 14.32it/s, est. speed input: 3341.31 toks/s, output: 869.51 toks/s]
Processed prompts:  59%|█████▉    | 5940/10000 [08:41<05:34, 12.13it/s, est. speed input: 3340.36 toks/s, output: 869.39 toks/s]
Processed prompts:  59%|█████▉    | 5944/10000 [08:41<06:26, 10.49it/s, est. speed input: 3339.01 toks/s, output: 869.20 toks/s]
Processed prompts:  60%|█████▉    | 5952/10000 [08:42<05:04, 13.30it/s, est. speed input: 3339.74 toks/s, output: 869.61 toks/s]
Processed prompts:  60%|█████▉    | 5954/10000 [08:42<05:22, 12.54it/s, est. speed input: 3339.28 toks/s, output: 869.60 toks/s]
Processed prompts:  60%|█████▉    | 5957/10000 [08:42<05:00, 13.47it/s, est. speed input: 3339.51 toks/s, output: 869.79 toks/s]
Processed prompts:  60%|█████▉    | 5961/10000 [08:43<04:54, 13.73it/s, est. speed input: 3339.49 toks/s, output: 869.95 toks/s]
Processed prompts:  60%|█████▉    | 5963/10000 [08:43<05:38, 11.94it/s, est. speed input: 3338.71 toks/s, output: 869.87 toks/s]
Processed prompts:  60%|█████▉    | 5965/10000 [08:43<05:55, 11.34it/s, est. speed input: 3338.25 toks/s, output: 869.81 toks/s]
Processed prompts:  60%|█████▉    | 5967/10000 [08:43<07:29,  8.97it/s, est. speed input: 3336.71 toks/s, output: 869.50 toks/s]
Processed prompts:  60%|█████▉    | 5969/10000 [08:44<08:08,  8.25it/s, est. speed input: 3335.66 toks/s, output: 869.34 toks/s]
Processed prompts:  60%|█████▉    | 5971/10000 [08:44<07:30,  8.94it/s, est. speed input: 3335.46 toks/s, output: 869.38 toks/s]
Processed prompts:  60%|█████▉    | 5977/10000 [08:44<04:46, 14.04it/s, est. speed input: 3336.77 toks/s, output: 869.88 toks/s]
Processed prompts:  60%|█████▉    | 5980/10000 [08:44<05:32, 12.11it/s, est. speed input: 3335.93 toks/s, output: 869.77 toks/s]
Processed prompts:  60%|█████▉    | 5984/10000 [08:45<05:05, 13.15it/s, est. speed input: 3336.06 toks/s, output: 869.93 toks/s]
Processed prompts:  60%|█████▉    | 5987/10000 [08:45<05:23, 12.39it/s, est. speed input: 3335.61 toks/s, output: 869.94 toks/s]
Processed prompts:  60%|█████▉    | 5990/10000 [08:45<05:26, 12.28it/s, est. speed input: 3335.36 toks/s, output: 869.96 toks/s]
Processed prompts:  60%|█████▉    | 5992/10000 [08:45<06:06, 10.93it/s, est. speed input: 3334.59 toks/s, output: 869.86 toks/s]
Processed prompts:  60%|█████▉    | 5998/10000 [08:46<04:18, 15.47it/s, est. speed input: 3335.93 toks/s, output: 870.44 toks/s]
Processed prompts:  60%|██████    | 6002/10000 [08:46<04:43, 14.12it/s, est. speed input: 3335.51 toks/s, output: 870.47 toks/s]
Processed prompts:  60%|██████    | 6008/10000 [08:46<04:02, 16.48it/s, est. speed input: 3336.48 toks/s, output: 871.00 toks/s]
Processed prompts:  60%|██████    | 6010/10000 [08:47<05:10, 12.84it/s, est. speed input: 3335.26 toks/s, output: 870.74 toks/s]
Processed prompts:  60%|██████    | 6013/10000 [08:47<05:01, 13.20it/s, est. speed input: 3335.25 toks/s, output: 870.84 toks/s]
Processed prompts:  60%|██████    | 6015/10000 [08:47<05:41, 11.68it/s, est. speed input: 3334.57 toks/s, output: 870.73 toks/s]
Processed prompts:  60%|██████    | 6018/10000 [08:47<05:22, 12.34it/s, est. speed input: 3334.54 toks/s, output: 870.82 toks/s]
Processed prompts:  60%|██████    | 6023/10000 [08:48<04:32, 14.60it/s, est. speed input: 3335.16 toks/s, output: 871.15 toks/s]
Processed prompts:  60%|██████    | 6025/10000 [08:48<05:41, 11.64it/s, est. speed input: 3334.04 toks/s, output: 870.93 toks/s]
Processed prompts:  60%|██████    | 6027/10000 [08:48<05:57, 11.12it/s, est. speed input: 3333.57 toks/s, output: 870.90 toks/s]
Processed prompts:  60%|██████    | 6032/10000 [08:48<04:33, 14.49it/s, est. speed input: 3334.43 toks/s, output: 871.27 toks/s]
Processed prompts:  60%|██████    | 6035/10000 [08:49<05:14, 12.62it/s, est. speed input: 3333.69 toks/s, output: 871.17 toks/s]
Processed prompts:  60%|██████    | 6037/10000 [08:49<05:53, 11.22it/s, est. speed input: 3332.96 toks/s, output: 871.04 toks/s]
Processed prompts:  60%|██████    | 6041/10000 [08:49<04:58, 13.26it/s, est. speed input: 3333.38 toks/s, output: 871.34 toks/s]
Processed prompts:  60%|██████    | 6044/10000 [08:49<05:18, 12.42it/s, est. speed input: 3332.97 toks/s, output: 871.35 toks/s]
Processed prompts:  60%|██████    | 6048/10000 [08:50<04:53, 13.45it/s, est. speed input: 3333.17 toks/s, output: 871.57 toks/s]
Processed prompts:  61%|██████    | 6052/10000 [08:50<04:46, 13.77it/s, est. speed input: 3333.16 toks/s, output: 871.77 toks/s]
Processed prompts:  61%|██████    | 6055/10000 [08:50<05:07, 12.85it/s, est. speed input: 3332.75 toks/s, output: 871.78 toks/s]
Processed prompts:  61%|██████    | 6057/10000 [08:50<05:47, 11.35it/s, est. speed input: 3332.01 toks/s, output: 871.64 toks/s]
Processed prompts:  61%|██████    | 6059/10000 [08:51<06:01, 10.91it/s, est. speed input: 3331.56 toks/s, output: 871.58 toks/s]
Processed prompts:  61%|██████    | 6061/10000 [08:51<06:15, 10.50it/s, est. speed input: 3331.08 toks/s, output: 871.54 toks/s]
Processed prompts:  61%|██████    | 6063/10000 [08:51<06:05, 10.76it/s, est. speed input: 3330.87 toks/s, output: 871.56 toks/s]
Processed prompts:  61%|██████    | 6065/10000 [08:51<06:19, 10.36it/s, est. speed input: 3330.40 toks/s, output: 871.51 toks/s]
Processed prompts:  61%|██████    | 6069/10000 [08:51<05:03, 12.93it/s, est. speed input: 3330.83 toks/s, output: 871.73 toks/s]
Processed prompts:  61%|██████    | 6072/10000 [08:52<05:32, 11.80it/s, est. speed input: 3330.20 toks/s, output: 871.69 toks/s]
Processed prompts:  61%|██████    | 6077/10000 [08:52<04:36, 14.21it/s, est. speed input: 3330.83 toks/s, output: 872.02 toks/s]
Processed prompts:  61%|██████    | 6081/10000 [08:52<04:46, 13.68it/s, est. speed input: 3330.56 toks/s, output: 872.09 toks/s]
Processed prompts:  61%|██████    | 6083/10000 [08:53<05:37, 11.61it/s, est. speed input: 3329.67 toks/s, output: 871.92 toks/s]
Processed prompts:  61%|██████    | 6086/10000 [08:53<05:19, 12.26it/s, est. speed input: 3329.68 toks/s, output: 872.00 toks/s]
Processed prompts:  61%|██████    | 6090/10000 [08:53<04:52, 13.35it/s, est. speed input: 3329.89 toks/s, output: 872.19 toks/s]
Processed prompts:  61%|██████    | 6094/10000 [08:53<04:46, 13.62it/s, est. speed input: 3329.87 toks/s, output: 872.29 toks/s]
Processed prompts:  61%|██████    | 6098/10000 [08:54<04:42, 13.82it/s, est. speed input: 3329.86 toks/s, output: 872.43 toks/s]
Processed prompts:  61%|██████    | 6101/10000 [08:54<05:03, 12.83it/s, est. speed input: 3329.42 toks/s, output: 872.43 toks/s]
Processed prompts:  61%|██████    | 6103/10000 [08:54<05:41, 11.42it/s, est. speed input: 3328.72 toks/s, output: 872.35 toks/s]
Processed prompts:  61%|██████    | 6108/10000 [08:54<04:27, 14.57it/s, est. speed input: 3329.54 toks/s, output: 872.73 toks/s]
Processed prompts:  61%|██████    | 6113/10000 [08:55<04:19, 14.95it/s, est. speed input: 3329.70 toks/s, output: 872.86 toks/s]
Processed prompts:  61%|██████    | 6115/10000 [08:55<05:15, 12.31it/s, est. speed input: 3328.71 toks/s, output: 872.66 toks/s]
Processed prompts:  61%|██████    | 6118/10000 [08:55<04:35, 14.09it/s, est. speed input: 3329.17 toks/s, output: 872.89 toks/s]
Processed prompts:  61%|██████    | 6122/10000 [08:55<04:25, 14.60it/s, est. speed input: 3329.30 toks/s, output: 873.08 toks/s]
Processed prompts:  61%|██████▏   | 6127/10000 [08:56<04:06, 15.72it/s, est. speed input: 3329.75 toks/s, output: 873.37 toks/s]
Processed prompts:  61%|██████▏   | 6134/10000 [08:56<03:35, 17.90it/s, est. speed input: 3330.75 toks/s, output: 873.83 toks/s]
Processed prompts:  61%|██████▏   | 6141/10000 [08:56<03:34, 18.03it/s, est. speed input: 3331.38 toks/s, output: 874.21 toks/s]
Processed prompts:  61%|██████▏   | 6146/10000 [08:57<03:56, 16.30it/s, est. speed input: 3331.20 toks/s, output: 874.29 toks/s]
Processed prompts:  62%|██████▏   | 6151/10000 [08:57<05:07, 12.53it/s, est. speed input: 3329.37 toks/s, output: 873.93 toks/s]
Processed prompts:  62%|██████▏   | 6155/10000 [08:58<05:19, 12.04it/s, est. speed input: 3328.75 toks/s, output: 873.87 toks/s]
Processed prompts:  62%|██████▏   | 6162/10000 [08:58<04:30, 14.19it/s, est. speed input: 3329.57 toks/s, output: 874.34 toks/s]
Processed prompts:  62%|██████▏   | 6165/10000 [08:58<04:48, 13.30it/s, est. speed input: 3329.07 toks/s, output: 874.33 toks/s]
Processed prompts:  62%|██████▏   | 6168/10000 [08:59<04:55, 12.97it/s, est. speed input: 3328.81 toks/s, output: 874.36 toks/s]
Processed prompts:  62%|██████▏   | 6170/10000 [08:59<05:25, 11.77it/s, est. speed input: 3328.10 toks/s, output: 874.31 toks/s]
Processed prompts:  62%|██████▏   | 6173/10000 [08:59<04:56, 12.89it/s, est. speed input: 3328.32 toks/s, output: 874.49 toks/s]
Processed prompts:  62%|██████▏   | 6176/10000 [08:59<05:13, 12.18it/s, est. speed input: 3327.92 toks/s, output: 874.46 toks/s]
Processed prompts:  62%|██████▏   | 6178/10000 [09:00<05:48, 10.97it/s, est. speed input: 3327.22 toks/s, output: 874.41 toks/s]
Processed prompts:  62%|██████▏   | 6183/10000 [09:00<04:16, 14.88it/s, est. speed input: 3328.26 toks/s, output: 874.87 toks/s]
Processed prompts:  62%|██████▏   | 6185/10000 [09:00<06:49,  9.31it/s, est. speed input: 3325.75 toks/s, output: 874.26 toks/s]
Processed prompts:  62%|██████▏   | 6188/10000 [09:00<05:52, 10.81it/s, est. speed input: 3326.00 toks/s, output: 874.38 toks/s]
Processed prompts:  62%|██████▏   | 6192/10000 [09:01<05:10, 12.26it/s, est. speed input: 3326.14 toks/s, output: 874.64 toks/s]
Processed prompts:  62%|██████▏   | 6194/10000 [09:01<05:52, 10.81it/s, est. speed input: 3325.34 toks/s, output: 874.55 toks/s]
Processed prompts:  62%|██████▏   | 6197/10000 [09:01<05:26, 11.64it/s, est. speed input: 3325.33 toks/s, output: 874.63 toks/s]
Processed prompts:  62%|██████▏   | 6200/10000 [09:01<05:22, 11.80it/s, est. speed input: 3325.10 toks/s, output: 874.68 toks/s]
Processed prompts:  62%|██████▏   | 6205/10000 [09:02<04:27, 14.19it/s, est. speed input: 3325.62 toks/s, output: 874.96 toks/s]
Processed prompts:  62%|██████▏   | 6211/10000 [09:02<03:56, 16.03it/s, est. speed input: 3326.29 toks/s, output: 875.35 toks/s]
Processed prompts:  62%|██████▏   | 6215/10000 [09:02<04:18, 14.66it/s, est. speed input: 3325.96 toks/s, output: 875.40 toks/s]
Processed prompts:  62%|██████▏   | 6217/10000 [09:03<05:02, 12.50it/s, est. speed input: 3325.16 toks/s, output: 875.27 toks/s]
Processed prompts:  62%|██████▏   | 6219/10000 [09:03<05:06, 12.34it/s, est. speed input: 3324.97 toks/s, output: 875.25 toks/s]
Processed prompts:  62%|██████▏   | 6221/10000 [09:03<05:26, 11.56it/s, est. speed input: 3324.50 toks/s, output: 875.25 toks/s]
Processed prompts:  62%|██████▏   | 6223/10000 [09:03<07:17,  8.64it/s, est. speed input: 3322.82 toks/s, output: 874.87 toks/s]
Processed prompts:  62%|██████▏   | 6226/10000 [09:04<06:01, 10.43it/s, est. speed input: 3323.08 toks/s, output: 875.00 toks/s]
Processed prompts:  62%|██████▏   | 6230/10000 [09:04<05:12, 12.07it/s, est. speed input: 3323.28 toks/s, output: 875.11 toks/s]
Processed prompts:  62%|██████▏   | 6235/10000 [09:04<04:31, 13.89it/s, est. speed input: 3323.65 toks/s, output: 875.40 toks/s]
Processed prompts:  62%|██████▏   | 6237/10000 [09:04<05:34, 11.27it/s, est. speed input: 3322.56 toks/s, output: 875.15 toks/s]
Processed prompts:  62%|██████▏   | 6239/10000 [09:05<05:46, 10.87it/s, est. speed input: 3322.13 toks/s, output: 875.12 toks/s]
Processed prompts:  62%|██████▏   | 6243/10000 [09:05<04:48, 13.01it/s, est. speed input: 3322.54 toks/s, output: 875.40 toks/s]
Processed prompts:  62%|██████▏   | 6245/10000 [09:05<06:46,  9.25it/s, est. speed input: 3320.67 toks/s, output: 875.01 toks/s]
Processed prompts:  62%|██████▏   | 6247/10000 [09:05<06:25,  9.75it/s, est. speed input: 3320.50 toks/s, output: 875.04 toks/s]
Processed prompts:  63%|██████▎   | 6251/10000 [09:06<05:10, 12.09it/s, est. speed input: 3320.88 toks/s, output: 875.35 toks/s]
Processed prompts:  63%|██████▎   | 6255/10000 [09:06<04:51, 12.84it/s, est. speed input: 3320.87 toks/s, output: 875.48 toks/s]
Processed prompts:  63%|██████▎   | 6259/10000 [09:06<04:40, 13.33it/s, est. speed input: 3320.89 toks/s, output: 875.61 toks/s]
Processed prompts:  63%|██████▎   | 6264/10000 [09:07<04:13, 14.73it/s, est. speed input: 3321.27 toks/s, output: 875.91 toks/s]
Processed prompts:  63%|██████▎   | 6267/10000 [09:07<05:43, 10.86it/s, est. speed input: 3319.50 toks/s, output: 875.54 toks/s]
Processed prompts:  63%|██████▎   | 6269/10000 [09:07<07:17,  8.53it/s, est. speed input: 3317.64 toks/s, output: 875.13 toks/s]
Processed prompts:  63%|██████▎   | 6274/10000 [09:08<06:13,  9.97it/s, est. speed input: 3317.45 toks/s, output: 875.27 toks/s]
Processed prompts:  63%|██████▎   | 6276/10000 [09:09<08:53,  6.98it/s, est. speed input: 3314.37 toks/s, output: 874.50 toks/s]
Processed prompts:  63%|██████▎   | 6281/10000 [09:09<07:26,  8.32it/s, est. speed input: 3313.93 toks/s, output: 874.50 toks/s]
Processed prompts:  63%|██████▎   | 6283/10000 [09:09<09:06,  6.80it/s, est. speed input: 3311.60 toks/s, output: 873.96 toks/s]
Processed prompts:  63%|██████▎   | 6289/10000 [09:10<06:33,  9.44it/s, est. speed input: 3312.18 toks/s, output: 874.31 toks/s]
Processed prompts:  63%|██████▎   | 6294/10000 [09:11<07:25,  8.32it/s, est. speed input: 3309.83 toks/s, output: 873.83 toks/s]
Processed prompts:  63%|██████▎   | 6299/10000 [09:11<07:24,  8.33it/s, est. speed input: 3308.38 toks/s, output: 873.62 toks/s]
Processed prompts:  63%|██████▎   | 6305/10000 [09:12<07:14,  8.51it/s, est. speed input: 3306.85 toks/s, output: 873.44 toks/s]
Processed prompts:  63%|██████▎   | 6306/10000 [09:12<10:01,  6.14it/s, est. speed input: 3303.16 toks/s, output: 872.51 toks/s]
Processed prompts:  63%|██████▎   | 6313/10000 [09:13<06:42,  9.16it/s, est. speed input: 3304.40 toks/s, output: 873.06 toks/s]
Processed prompts:  63%|██████▎   | 6315/10000 [09:13<08:53,  6.91it/s, est. speed input: 3301.17 toks/s, output: 872.32 toks/s]
Processed prompts:  63%|██████▎   | 6317/10000 [09:14<09:45,  6.29it/s, est. speed input: 3299.34 toks/s, output: 871.87 toks/s]
Processed prompts:  63%|██████▎   | 6319/10000 [09:14<10:36,  5.78it/s, est. speed input: 3297.48 toks/s, output: 871.42 toks/s]
Processed prompts:  63%|██████▎   | 6324/10000 [09:15<07:55,  7.72it/s, est. speed input: 3297.53 toks/s, output: 871.54 toks/s]
Processed prompts:  63%|██████▎   | 6329/10000 [09:15<07:29,  8.17it/s, est. speed input: 3296.34 toks/s, output: 871.32 toks/s]
Processed prompts:  63%|██████▎   | 6330/10000 [09:16<10:19,  5.92it/s, est. speed input: 3293.18 toks/s, output: 870.49 toks/s]
Processed prompts:  63%|██████▎   | 6331/10000 [09:16<11:57,  5.11it/s, est. speed input: 3291.29 toks/s, output: 870.02 toks/s]
Processed prompts:  63%|██████▎   | 6333/10000 [09:17<10:45,  5.68it/s, est. speed input: 3290.69 toks/s, output: 869.90 toks/s]
Processed prompts:  63%|██████▎   | 6336/10000 [09:17<09:19,  6.55it/s, est. speed input: 3289.93 toks/s, output: 869.79 toks/s]
Processed prompts:  63%|██████▎   | 6339/10000 [09:18<12:46,  4.78it/s, est. speed input: 3285.51 toks/s, output: 868.67 toks/s]
Processed prompts:  63%|██████▎   | 6340/10000 [09:18<14:46,  4.13it/s, est. speed input: 3283.30 toks/s, output: 868.14 toks/s]
Processed prompts:  63%|██████▎   | 6348/10000 [09:19<07:55,  7.68it/s, est. speed input: 3283.81 toks/s, output: 868.46 toks/s]
Processed prompts:  64%|██████▎   | 6350/10000 [09:19<10:14,  5.94it/s, est. speed input: 3280.64 toks/s, output: 867.67 toks/s]
Processed prompts:  64%|██████▎   | 6352/10000 [09:20<10:50,  5.61it/s, est. speed input: 3278.94 toks/s, output: 867.24 toks/s]
Processed prompts:  64%|██████▎   | 6353/10000 [09:20<11:41,  5.20it/s, est. speed input: 3277.67 toks/s, output: 866.99 toks/s]
Processed prompts:  64%|██████▎   | 6358/10000 [09:20<07:38,  7.94it/s, est. speed input: 3278.23 toks/s, output: 867.29 toks/s]
Processed prompts:  64%|██████▎   | 6362/10000 [09:21<07:52,  7.70it/s, est. speed input: 3276.68 toks/s, output: 867.02 toks/s]
Processed prompts:  64%|██████▎   | 6370/10000 [09:22<06:11,  9.78it/s, est. speed input: 3276.48 toks/s, output: 867.25 toks/s]
Processed prompts:  64%|██████▎   | 6372/10000 [09:22<09:05,  6.65it/s, est. speed input: 3272.40 toks/s, output: 866.27 toks/s]
Processed prompts:  64%|██████▍   | 6375/10000 [09:23<08:47,  6.87it/s, est. speed input: 3271.34 toks/s, output: 866.12 toks/s]
Processed prompts:  64%|██████▍   | 6381/10000 [09:23<07:01,  8.58it/s, est. speed input: 3271.19 toks/s, output: 866.29 toks/s]
Processed prompts:  64%|██████▍   | 6384/10000 [09:24<09:02,  6.67it/s, est. speed input: 3267.85 toks/s, output: 865.55 toks/s]
Processed prompts:  64%|██████▍   | 6390/10000 [09:25<07:38,  7.86it/s, est. speed input: 3267.14 toks/s, output: 865.48 toks/s]
Processed prompts:  64%|██████▍   | 6391/10000 [09:25<10:49,  5.56it/s, est. speed input: 3263.28 toks/s, output: 864.51 toks/s]
Processed prompts:  64%|██████▍   | 6393/10000 [09:26<11:02,  5.44it/s, est. speed input: 3261.80 toks/s, output: 864.16 toks/s]
Processed prompts:  64%|██████▍   | 6395/10000 [09:26<10:31,  5.70it/s, est. speed input: 3260.98 toks/s, output: 864.02 toks/s]
Processed prompts:  64%|██████▍   | 6398/10000 [09:26<09:35,  6.26it/s, est. speed input: 3259.99 toks/s, output: 863.83 toks/s]
Processed prompts:  64%|██████▍   | 6403/10000 [09:27<08:18,  7.22it/s, est. speed input: 3258.82 toks/s, output: 863.69 toks/s]
Processed prompts:  64%|██████▍   | 6408/10000 [09:28<07:51,  7.61it/s, est. speed input: 3257.48 toks/s, output: 863.53 toks/s]
Processed prompts:  64%|██████▍   | 6409/10000 [09:28<11:07,  5.38it/s, est. speed input: 3253.97 toks/s, output: 862.65 toks/s]
Processed prompts:  64%|██████▍   | 6411/10000 [09:29<10:48,  5.53it/s, est. speed input: 3252.96 toks/s, output: 862.45 toks/s]
Processed prompts:  64%|██████▍   | 6417/10000 [09:29<07:25,  8.03it/s, est. speed input: 3253.16 toks/s, output: 862.71 toks/s]
Processed prompts:  64%|██████▍   | 6421/10000 [09:30<08:18,  7.17it/s, est. speed input: 3250.85 toks/s, output: 862.29 toks/s]
Processed prompts:  64%|██████▍   | 6422/10000 [09:30<11:41,  5.10it/s, est. speed input: 3247.30 toks/s, output: 861.43 toks/s]
Processed prompts:  64%|██████▍   | 6424/10000 [09:31<11:22,  5.24it/s, est. speed input: 3246.13 toks/s, output: 861.22 toks/s]
Processed prompts:  64%|██████▍   | 6430/10000 [09:31<07:19,  8.13it/s, est. speed input: 3246.81 toks/s, output: 861.60 toks/s]
Processed prompts:  64%|██████▍   | 6434/10000 [09:32<08:43,  6.81it/s, est. speed input: 3243.95 toks/s, output: 860.94 toks/s]
Processed prompts:  64%|██████▍   | 6440/10000 [09:32<07:48,  7.60it/s, est. speed input: 3242.63 toks/s, output: 860.79 toks/s]
Processed prompts:  64%|██████▍   | 6442/10000 [09:33<09:47,  6.05it/s, est. speed input: 3239.57 toks/s, output: 860.11 toks/s]
Processed prompts:  64%|██████▍   | 6443/10000 [09:34<11:19,  5.24it/s, est. speed input: 3237.64 toks/s, output: 859.64 toks/s]
Processed prompts:  64%|██████▍   | 6446/10000 [09:34<09:36,  6.16it/s, est. speed input: 3237.20 toks/s, output: 859.65 toks/s]
Processed prompts:  64%|██████▍   | 6449/10000 [09:34<09:03,  6.53it/s, est. speed input: 3236.19 toks/s, output: 859.49 toks/s]
Processed prompts:  65%|██████▍   | 6453/10000 [09:35<08:10,  7.23it/s, est. speed input: 3235.28 toks/s, output: 859.38 toks/s]
Processed prompts:  65%|██████▍   | 6458/10000 [09:35<07:32,  7.82it/s, est. speed input: 3234.18 toks/s, output: 859.28 toks/s]
Processed prompts:  65%|██████▍   | 6462/10000 [09:36<07:53,  7.47it/s, est. speed input: 3232.45 toks/s, output: 858.94 toks/s]
Processed prompts:  65%|██████▍   | 6465/10000 [09:36<08:39,  6.80it/s, est. speed input: 3230.52 toks/s, output: 858.51 toks/s]
Processed prompts:  65%|██████▍   | 6467/10000 [09:37<09:14,  6.37it/s, est. speed input: 3229.09 toks/s, output: 858.20 toks/s]
Processed prompts:  65%|██████▍   | 6469/10000 [09:37<09:30,  6.19it/s, est. speed input: 3227.91 toks/s, output: 857.95 toks/s]
Processed prompts:  65%|██████▍   | 6471/10000 [09:37<09:40,  6.08it/s, est. speed input: 3226.77 toks/s, output: 857.70 toks/s]
Processed prompts:  65%|██████▍   | 6473/10000 [09:38<10:03,  5.85it/s, est. speed input: 3225.48 toks/s, output: 857.42 toks/s]
Processed prompts:  65%|██████▍   | 6476/10000 [09:38<10:42,  5.49it/s, est. speed input: 3223.26 toks/s, output: 856.93 toks/s]
Processed prompts:  65%|██████▍   | 6477/10000 [09:39<10:53,  5.39it/s, est. speed input: 3222.51 toks/s, output: 856.77 toks/s]
Processed prompts:  65%|██████▍   | 6481/10000 [09:39<07:41,  7.62it/s, est. speed input: 3222.72 toks/s, output: 856.93 toks/s]
Processed prompts:  65%|██████▍   | 6485/10000 [09:39<07:43,  7.59it/s, est. speed input: 3221.38 toks/s, output: 856.66 toks/s]
Processed prompts:  65%|██████▍   | 6487/10000 [09:40<09:50,  5.95it/s, est. speed input: 3218.87 toks/s, output: 856.07 toks/s]
Processed prompts:  65%|██████▍   | 6488/10000 [09:41<14:19,  4.09it/s, est. speed input: 3215.20 toks/s, output: 855.16 toks/s]
Processed prompts:  65%|██████▍   | 6493/10000 [09:41<08:20,  7.00it/s, est. speed input: 3216.19 toks/s, output: 855.57 toks/s]
Processed prompts:  65%|██████▍   | 6495/10000 [09:42<09:59,  5.85it/s, est. speed input: 3214.76 toks/s, output: 855.01 toks/s]
Processed prompts:  65%|██████▍   | 6496/10000 [09:42<11:51,  4.93it/s, est. speed input: 3212.88 toks/s, output: 854.55 toks/s]
Processed prompts:  65%|██████▍   | 6497/10000 [09:42<13:19,  4.38it/s, est. speed input: 3211.28 toks/s, output: 854.21 toks/s]
Processed prompts:  65%|██████▌   | 6501/10000 [09:43<08:54,  6.54it/s, est. speed input: 3211.29 toks/s, output: 854.37 toks/s]
Processed prompts:  65%|██████▌   | 6504/10000 [09:43<09:35,  6.08it/s, est. speed input: 3209.96 toks/s, output: 853.94 toks/s]
Processed prompts:  65%|██████▌   | 6506/10000 [09:44<11:04,  5.26it/s, est. speed input: 3208.23 toks/s, output: 853.44 toks/s]
Processed prompts:  65%|██████▌   | 6510/10000 [09:44<08:52,  6.55it/s, est. speed input: 3208.29 toks/s, output: 853.47 toks/s]
Processed prompts:  65%|██████▌   | 6511/10000 [09:45<11:19,  5.13it/s, est. speed input: 3206.03 toks/s, output: 852.92 toks/s]
Processed prompts:  65%|██████▌   | 6514/10000 [09:45<09:23,  6.18it/s, est. speed input: 3205.53 toks/s, output: 852.93 toks/s]
Processed prompts:  65%|██████▌   | 6515/10000 [09:45<11:50,  4.91it/s, est. speed input: 3204.04 toks/s, output: 852.41 toks/s]
Processed prompts:  65%|██████▌   | 6520/10000 [09:46<07:51,  7.38it/s, est. speed input: 3206.44 toks/s, output: 852.61 toks/s]
Processed prompts:  65%|██████▌   | 6522/10000 [09:47<12:51,  4.51it/s, est. speed input: 3201.55 toks/s, output: 851.43 toks/s]
Processed prompts:  65%|██████▌   | 6524/10000 [09:47<10:50,  5.34it/s, est. speed input: 3201.75 toks/s, output: 851.49 toks/s]
Processed prompts:  65%|██████▌   | 6527/10000 [09:47<09:16,  6.25it/s, est. speed input: 3204.94 toks/s, output: 851.43 toks/s]
Processed prompts:  65%|██████▌   | 6528/10000 [09:48<11:53,  4.87it/s, est. speed input: 3203.03 toks/s, output: 850.88 toks/s]
Processed prompts:  65%|██████▌   | 6531/10000 [09:48<09:54,  5.84it/s, est. speed input: 3204.21 toks/s, output: 850.81 toks/s]
Processed prompts:  65%|██████▌   | 6535/10000 [09:48<08:29,  6.80it/s, est. speed input: 3205.28 toks/s, output: 850.76 toks/s]
Processed prompts:  65%|██████▌   | 6541/10000 [09:49<07:12,  7.99it/s, est. speed input: 3209.99 toks/s, output: 850.74 toks/s]
Processed prompts:  65%|██████▌   | 6545/10000 [09:50<07:57,  7.24it/s, est. speed input: 3211.42 toks/s, output: 850.35 toks/s]
Processed prompts:  65%|██████▌   | 6548/10000 [09:50<08:51,  6.50it/s, est. speed input: 3210.92 toks/s, output: 849.91 toks/s]
Processed prompts:  66%|██████▌   | 6550/10000 [09:51<09:29,  6.06it/s, est. speed input: 3211.83 toks/s, output: 849.61 toks/s]
Processed prompts:  66%|██████▌   | 6558/10000 [09:51<05:48,  9.87it/s, est. speed input: 3219.37 toks/s, output: 850.21 toks/s]
Processed prompts:  66%|██████▌   | 6563/10000 [09:52<06:51,  8.36it/s, est. speed input: 3220.59 toks/s, output: 849.77 toks/s]
Processed prompts:  66%|██████▌   | 6565/10000 [09:53<10:13,  5.60it/s, est. speed input: 3217.78 toks/s, output: 848.61 toks/s]
Processed prompts:  66%|██████▌   | 6568/10000 [09:53<09:12,  6.21it/s, est. speed input: 3218.89 toks/s, output: 848.60 toks/s]
Processed prompts:  66%|██████▌   | 6572/10000 [09:54<08:29,  6.73it/s, est. speed input: 3220.56 toks/s, output: 848.49 toks/s]
Processed prompts:  66%|██████▌   | 6577/10000 [09:54<08:32,  6.68it/s, est. speed input: 3223.04 toks/s, output: 848.12 toks/s]
Processed prompts:  66%|██████▌   | 6579/10000 [09:55<10:23,  5.49it/s, est. speed input: 3221.65 toks/s, output: 847.47 toks/s]
Processed prompts:  66%|██████▌   | 6583/10000 [09:56<09:05,  6.26it/s, est. speed input: 3222.90 toks/s, output: 847.38 toks/s]
Processed prompts:  66%|██████▌   | 6585/10000 [09:56<09:54,  5.75it/s, est. speed input: 3222.23 toks/s, output: 847.01 toks/s]
Processed prompts:  66%|██████▌   | 6587/10000 [09:56<10:11,  5.58it/s, est. speed input: 3222.29 toks/s, output: 846.74 toks/s]
Processed prompts:  66%|██████▌   | 6588/10000 [09:57<11:26,  4.97it/s, est. speed input: 3221.64 toks/s, output: 846.39 toks/s]
Processed prompts:  66%|██████▌   | 6591/10000 [09:57<09:10,  6.19it/s, est. speed input: 3222.70 toks/s, output: 846.42 toks/s]
Processed prompts:  66%|██████▌   | 6592/10000 [09:57<11:20,  5.01it/s, est. speed input: 3221.60 toks/s, output: 845.97 toks/s]
Processed prompts:  66%|██████▌   | 6595/10000 [09:58<09:27,  6.00it/s, est. speed input: 3222.70 toks/s, output: 845.91 toks/s]
Processed prompts:  66%|██████▌   | 6600/10000 [09:58<07:22,  7.68it/s, est. speed input: 3226.45 toks/s, output: 845.89 toks/s]
Processed prompts:  66%|██████▌   | 6601/10000 [09:59<10:19,  5.49it/s, est. speed input: 3224.95 toks/s, output: 845.26 toks/s]
Processed prompts:  66%|██████▌   | 6605/10000 [09:59<07:52,  7.19it/s, est. speed input: 3226.93 toks/s, output: 845.40 toks/s]
Processed prompts:  66%|██████▌   | 6609/10000 [10:00<07:18,  7.73it/s, est. speed input: 3228.95 toks/s, output: 845.35 toks/s]
Processed prompts:  66%|██████▌   | 6613/10000 [10:00<08:00,  7.06it/s, est. speed input: 3231.94 toks/s, output: 844.98 toks/s]
Processed prompts:  66%|██████▌   | 6620/10000 [10:02<09:50,  5.73it/s, est. speed input: 3233.31 toks/s, output: 843.78 toks/s]
Processed prompts:  66%|██████▌   | 6621/10000 [10:02<10:22,  5.43it/s, est. speed input: 3233.50 toks/s, output: 843.53 toks/s]
Processed prompts:  66%|██████▌   | 6624/10000 [10:02<08:43,  6.44it/s, est. speed input: 3235.89 toks/s, output: 843.63 toks/s]
Processed prompts:  66%|██████▋   | 6627/10000 [10:03<08:21,  6.73it/s, est. speed input: 3236.72 toks/s, output: 843.50 toks/s]
Processed prompts:  66%|██████▋   | 6628/10000 [10:03<10:33,  5.32it/s, est. speed input: 3235.60 toks/s, output: 842.95 toks/s]
Processed prompts:  66%|██████▋   | 6629/10000 [10:03<11:21,  4.95it/s, est. speed input: 3235.11 toks/s, output: 842.72 toks/s]
Processed prompts:  66%|██████▋   | 6634/10000 [10:04<07:06,  7.89it/s, est. speed input: 3239.38 toks/s, output: 842.99 toks/s]
Processed prompts:  66%|██████▋   | 6635/10000 [10:04<10:24,  5.38it/s, est. speed input: 3237.26 toks/s, output: 842.35 toks/s]
Processed prompts:  66%|██████▋   | 6638/10000 [10:05<09:41,  5.78it/s, est. speed input: 3238.73 toks/s, output: 842.14 toks/s]
Processed prompts:  66%|██████▋   | 6641/10000 [10:05<08:59,  6.23it/s, est. speed input: 3240.05 toks/s, output: 842.00 toks/s]
Processed prompts:  66%|██████▋   | 6645/10000 [10:06<08:26,  6.62it/s, est. speed input: 3241.29 toks/s, output: 841.86 toks/s]
Processed prompts:  66%|██████▋   | 6648/10000 [10:06<09:06,  6.13it/s, est. speed input: 3241.38 toks/s, output: 841.48 toks/s]
Processed prompts:  67%|██████▋   | 6653/10000 [10:07<07:25,  7.52it/s, est. speed input: 3244.19 toks/s, output: 841.54 toks/s]
Processed prompts:  67%|██████▋   | 6656/10000 [10:07<08:28,  6.57it/s, est. speed input: 3242.72 toks/s, output: 841.09 toks/s]
Processed prompts:  67%|██████▋   | 6658/10000 [10:08<09:19,  5.97it/s, est. speed input: 3243.64 toks/s, output: 840.75 toks/s]
Processed prompts:  67%|██████▋   | 6662/10000 [10:08<07:40,  7.25it/s, est. speed input: 3246.67 toks/s, output: 840.84 toks/s]
Processed prompts:  67%|██████▋   | 6667/10000 [10:09<07:07,  7.79it/s, est. speed input: 3249.20 toks/s, output: 840.75 toks/s]
Processed prompts:  67%|██████▋   | 6672/10000 [10:09<07:11,  7.70it/s, est. speed input: 3251.33 toks/s, output: 840.53 toks/s]
Processed prompts:  67%|██████▋   | 6673/10000 [10:10<10:14,  5.41it/s, est. speed input: 3248.58 toks/s, output: 839.68 toks/s]
Processed prompts:  67%|██████▋   | 6676/10000 [10:10<08:58,  6.17it/s, est. speed input: 3250.08 toks/s, output: 839.65 toks/s]
Processed prompts:  67%|██████▋   | 6682/10000 [10:11<06:57,  7.94it/s, est. speed input: 3253.80 toks/s, output: 839.85 toks/s]
Processed prompts:  67%|██████▋   | 6684/10000 [10:12<09:06,  6.07it/s, est. speed input: 3252.38 toks/s, output: 839.18 toks/s]
Processed prompts:  67%|██████▋   | 6686/10000 [10:12<10:28,  5.27it/s, est. speed input: 3251.83 toks/s, output: 838.69 toks/s]
Processed prompts:  67%|██████▋   | 6689/10000 [10:13<09:11,  6.00it/s, est. speed input: 3253.56 toks/s, output: 838.64 toks/s]
Processed prompts:  67%|██████▋   | 6692/10000 [10:13<08:47,  6.28it/s, est. speed input: 3255.18 toks/s, output: 838.52 toks/s]
Processed prompts:  67%|██████▋   | 6696/10000 [10:13<07:51,  7.00it/s, est. speed input: 3256.82 toks/s, output: 838.43 toks/s]
Processed prompts:  67%|██████▋   | 6701/10000 [10:14<07:00,  7.84it/s, est. speed input: 3260.38 toks/s, output: 838.45 toks/s]
Processed prompts:  67%|██████▋   | 6703/10000 [10:15<09:08,  6.01it/s, est. speed input: 3258.64 toks/s, output: 837.84 toks/s]
Processed prompts:  67%|██████▋   | 6706/10000 [10:15<09:59,  5.49it/s, est. speed input: 3259.43 toks/s, output: 837.38 toks/s]
Processed prompts:  67%|██████▋   | 6711/10000 [10:16<07:44,  7.08it/s, est. speed input: 3264.33 toks/s, output: 837.51 toks/s]
Processed prompts:  67%|██████▋   | 6712/10000 [10:16<10:12,  5.37it/s, est. speed input: 3262.40 toks/s, output: 836.87 toks/s]
Processed prompts:  67%|██████▋   | 6715/10000 [10:17<09:22,  5.84it/s, est. speed input: 3263.64 toks/s, output: 836.75 toks/s]
Processed prompts:  67%|██████▋   | 6716/10000 [10:17<10:56,  5.00it/s, est. speed input: 3262.47 toks/s, output: 836.34 toks/s]
Processed prompts:  67%|██████▋   | 6718/10000 [10:17<10:13,  5.35it/s, est. speed input: 3262.48 toks/s, output: 836.19 toks/s]
Processed prompts:  67%|██████▋   | 6721/10000 [10:18<09:10,  5.96it/s, est. speed input: 3263.45 toks/s, output: 836.04 toks/s]
Processed prompts:  67%|██████▋   | 6722/10000 [10:18<11:30,  4.75it/s, est. speed input: 3262.43 toks/s, output: 835.58 toks/s]
Processed prompts:  67%|██████▋   | 6725/10000 [10:19<09:11,  5.94it/s, est. speed input: 3264.40 toks/s, output: 835.62 toks/s]
Processed prompts:  67%|██████▋   | 6730/10000 [10:19<07:19,  7.43it/s, est. speed input: 3265.63 toks/s, output: 835.64 toks/s]
Processed prompts:  67%|██████▋   | 6731/10000 [10:20<10:51,  5.02it/s, est. speed input: 3263.26 toks/s, output: 834.89 toks/s]
Processed prompts:  67%|██████▋   | 6733/10000 [10:20<10:54,  4.99it/s, est. speed input: 3262.59 toks/s, output: 834.65 toks/s]
Processed prompts:  67%|██████▋   | 6734/10000 [10:20<11:56,  4.56it/s, est. speed input: 3261.71 toks/s, output: 834.38 toks/s]
Processed prompts:  67%|██████▋   | 6740/10000 [10:21<06:43,  8.08it/s, est. speed input: 3267.15 toks/s, output: 834.81 toks/s]
Processed prompts:  67%|██████▋   | 6742/10000 [10:21<09:18,  5.83it/s, est. speed input: 3265.90 toks/s, output: 834.13 toks/s]
Processed prompts:  67%|██████▋   | 6744/10000 [10:22<10:27,  5.19it/s, est. speed input: 3265.89 toks/s, output: 833.73 toks/s]
Processed prompts:  67%|██████▋   | 6747/10000 [10:22<09:24,  5.77it/s, est. speed input: 3266.63 toks/s, output: 833.64 toks/s]
Processed prompts:  67%|██████▋   | 6749/10000 [10:23<09:49,  5.51it/s, est. speed input: 3267.25 toks/s, output: 833.34 toks/s]
Processed prompts:  68%|██████▊   | 6750/10000 [10:23<11:46,  4.60it/s, est. speed input: 3266.46 toks/s, output: 832.92 toks/s]
Processed prompts:  68%|██████▊   | 6755/10000 [10:23<07:15,  7.45it/s, est. speed input: 3270.55 toks/s, output: 833.19 toks/s]
Processed prompts:  68%|██████▊   | 6756/10000 [10:24<10:56,  4.94it/s, est. speed input: 3268.74 toks/s, output: 832.42 toks/s]
Processed prompts:  68%|██████▊   | 6759/10000 [10:25<10:03,  5.37it/s, est. speed input: 3270.11 toks/s, output: 832.17 toks/s]
Processed prompts:  68%|██████▊   | 6760/10000 [10:25<11:46,  4.58it/s, est. speed input: 3269.22 toks/s, output: 831.78 toks/s]
Processed prompts:  68%|██████▊   | 6766/10000 [10:25<07:13,  7.45it/s, est. speed input: 3274.46 toks/s, output: 832.09 toks/s]
Processed prompts:  68%|██████▊   | 6770/10000 [10:26<07:50,  6.86it/s, est. speed input: 3274.50 toks/s, output: 831.72 toks/s]
Processed prompts:  68%|██████▊   | 6773/10000 [10:27<08:29,  6.34it/s, est. speed input: 3274.68 toks/s, output: 831.35 toks/s]
Processed prompts:  68%|██████▊   | 6774/10000 [10:28<12:54,  4.17it/s, est. speed input: 3271.02 toks/s, output: 830.34 toks/s]
Processed prompts:  68%|██████▊   | 6779/10000 [10:28<09:28,  5.66it/s, est. speed input: 3273.28 toks/s, output: 830.34 toks/s]
Processed prompts:  68%|██████▊   | 6782/10000 [10:28<08:51,  6.06it/s, est. speed input: 3274.21 toks/s, output: 830.18 toks/s]
Processed prompts:  68%|██████▊   | 6785/10000 [10:29<08:44,  6.13it/s, est. speed input: 3274.71 toks/s, output: 829.95 toks/s]
Processed prompts:  68%|██████▊   | 6790/10000 [10:29<07:17,  7.33it/s, est. speed input: 3278.99 toks/s, output: 829.93 toks/s]
Processed prompts:  68%|██████▊   | 6791/10000 [10:30<10:16,  5.21it/s, est. speed input: 3277.00 toks/s, output: 829.22 toks/s]
Processed prompts:  68%|██████▊   | 6796/10000 [10:30<07:17,  7.32it/s, est. speed input: 3278.59 toks/s, output: 829.48 toks/s]
Processed prompts:  68%|██████▊   | 6799/10000 [10:31<08:36,  6.19it/s, est. speed input: 3278.31 toks/s, output: 828.96 toks/s]
Processed prompts:  68%|██████▊   | 6805/10000 [10:32<07:03,  7.54it/s, est. speed input: 3282.38 toks/s, output: 829.00 toks/s]
Processed prompts:  68%|██████▊   | 6806/10000 [10:32<09:38,  5.52it/s, est. speed input: 3280.25 toks/s, output: 828.29 toks/s]
Processed prompts:  68%|██████▊   | 6810/10000 [10:33<07:34,  7.03it/s, est. speed input: 3284.03 toks/s, output: 828.45 toks/s]
Processed prompts:  68%|██████▊   | 6812/10000 [10:33<08:50,  6.01it/s, est. speed input: 3283.12 toks/s, output: 828.04 toks/s]
Processed prompts:  68%|██████▊   | 6816/10000 [10:33<07:19,  7.24it/s, est. speed input: 3286.42 toks/s, output: 828.11 toks/s]
Processed prompts:  68%|██████▊   | 6821/10000 [10:34<07:16,  7.28it/s, est. speed input: 3288.89 toks/s, output: 828.00 toks/s]
Processed prompts:  68%|██████▊   | 6828/10000 [10:36<08:46,  6.02it/s, est. speed input: 3288.99 toks/s, output: 827.07 toks/s]
Processed prompts:  68%|██████▊   | 6831/10000 [10:36<08:22,  6.30it/s, est. speed input: 3292.96 toks/s, output: 826.96 toks/s]
Processed prompts:  68%|██████▊   | 6833/10000 [10:36<08:44,  6.04it/s, est. speed input: 3293.93 toks/s, output: 826.71 toks/s]
Processed prompts:  68%|██████▊   | 6834/10000 [10:37<10:02,  5.26it/s, est. speed input: 3292.99 toks/s, output: 826.33 toks/s]
Processed prompts:  68%|██████▊   | 6837/10000 [10:37<08:27,  6.24it/s, est. speed input: 3294.43 toks/s, output: 826.33 toks/s]
Processed prompts:  68%|██████▊   | 6840/10000 [10:37<08:29,  6.21it/s, est. speed input: 3295.50 toks/s, output: 826.10 toks/s]
Processed prompts:  68%|██████▊   | 6845/10000 [10:38<07:12,  7.29it/s, est. speed input: 3298.66 toks/s, output: 826.08 toks/s]
Processed prompts:  68%|██████▊   | 6850/10000 [10:39<07:15,  7.24it/s, est. speed input: 3299.66 toks/s, output: 825.84 toks/s]
Processed prompts:  69%|██████▊   | 6853/10000 [10:39<08:15,  6.35it/s, est. speed input: 3300.97 toks/s, output: 825.40 toks/s]
Processed prompts:  69%|██████▊   | 6854/10000 [10:40<10:21,  5.06it/s, est. speed input: 3299.75 toks/s, output: 824.86 toks/s]
Processed prompts:  69%|██████▊   | 6858/10000 [10:40<08:04,  6.48it/s, est. speed input: 3301.07 toks/s, output: 824.96 toks/s]
Processed prompts:  69%|██████▊   | 6861/10000 [10:41<08:05,  6.46it/s, est. speed input: 3301.01 toks/s, output: 824.73 toks/s]
Processed prompts:  69%|██████▊   | 6863/10000 [10:41<09:46,  5.35it/s, est. speed input: 3299.87 toks/s, output: 824.24 toks/s]
Processed prompts:  69%|██████▊   | 6865/10000 [10:42<10:12,  5.12it/s, est. speed input: 3299.84 toks/s, output: 823.96 toks/s]
Processed prompts:  69%|██████▊   | 6868/10000 [10:42<09:05,  5.74it/s, est. speed input: 3301.68 toks/s, output: 823.86 toks/s]
Processed prompts:  69%|██████▊   | 6872/10000 [10:43<07:49,  6.66it/s, est. speed input: 3303.43 toks/s, output: 823.76 toks/s]
Processed prompts:  69%|██████▊   | 6873/10000 [10:43<10:26,  5.00it/s, est. speed input: 3301.52 toks/s, output: 823.20 toks/s]
Processed prompts:  69%|██████▉   | 6878/10000 [10:44<07:29,  6.95it/s, est. speed input: 3305.32 toks/s, output: 823.37 toks/s]
Processed prompts:  69%|██████▉   | 6881/10000 [10:44<08:36,  6.04it/s, est. speed input: 3304.52 toks/s, output: 822.92 toks/s]
Processed prompts:  69%|██████▉   | 6882/10000 [10:45<11:10,  4.65it/s, est. speed input: 3302.31 toks/s, output: 822.35 toks/s]
Processed prompts:  69%|██████▉   | 6885/10000 [10:45<09:10,  5.65it/s, est. speed input: 3304.56 toks/s, output: 822.36 toks/s]
Processed prompts:  69%|██████▉   | 6886/10000 [10:46<11:23,  4.56it/s, est. speed input: 3302.69 toks/s, output: 821.92 toks/s]
Processed prompts:  69%|██████▉   | 6888/10000 [10:46<10:07,  5.12it/s, est. speed input: 3303.01 toks/s, output: 821.85 toks/s]
Processed prompts:  69%|██████▉   | 6893/10000 [10:46<06:49,  7.60it/s, est. speed input: 3306.30 toks/s, output: 822.06 toks/s]
Processed prompts:  69%|██████▉   | 6901/10000 [10:47<05:13,  9.90it/s, est. speed input: 3310.97 toks/s, output: 822.30 toks/s]
Processed prompts:  69%|██████▉   | 6903/10000 [10:48<07:54,  6.52it/s, est. speed input: 3308.98 toks/s, output: 821.50 toks/s]
Processed prompts:  69%|██████▉   | 6907/10000 [10:48<07:20,  7.02it/s, est. speed input: 3311.34 toks/s, output: 821.41 toks/s]
Processed prompts:  69%|██████▉   | 6909/10000 [10:49<08:06,  6.35it/s, est. speed input: 3310.75 toks/s, output: 821.11 toks/s]
Processed prompts:  69%|██████▉   | 6913/10000 [10:49<06:47,  7.58it/s, est. speed input: 3312.50 toks/s, output: 821.22 toks/s]
Processed prompts:  69%|██████▉   | 6914/10000 [10:49<09:11,  5.59it/s, est. speed input: 3311.38 toks/s, output: 820.64 toks/s]
Processed prompts:  69%|██████▉   | 6915/10000 [10:50<10:42,  4.80it/s, est. speed input: 3310.46 toks/s, output: 820.26 toks/s]
Processed prompts:  69%|██████▉   | 6916/10000 [10:50<11:17,  4.55it/s, est. speed input: 3310.36 toks/s, output: 820.07 toks/s]
Processed prompts:  69%|██████▉   | 6921/10000 [10:50<07:02,  7.28it/s, est. speed input: 3314.43 toks/s, output: 820.32 toks/s]
Processed prompts:  69%|██████▉   | 6924/10000 [10:51<08:10,  6.27it/s, est. speed input: 3314.25 toks/s, output: 819.95 toks/s]
Processed prompts:  69%|██████▉   | 6929/10000 [10:52<06:47,  7.54it/s, est. speed input: 3317.77 toks/s, output: 819.95 toks/s]
Processed prompts:  69%|██████▉   | 6930/10000 [10:52<09:22,  5.46it/s, est. speed input: 3315.63 toks/s, output: 819.34 toks/s]
Processed prompts:  69%|██████▉   | 6932/10000 [10:53<09:43,  5.25it/s, est. speed input: 3315.13 toks/s, output: 819.05 toks/s]
Processed prompts:  69%|██████▉   | 6933/10000 [10:53<13:29,  3.79it/s, est. speed input: 3312.62 toks/s, output: 818.31 toks/s]
Processed prompts:  69%|██████▉   | 6937/10000 [10:53<08:27,  6.04it/s, est. speed input: 3314.62 toks/s, output: 818.60 toks/s]
Processed prompts:  69%|██████▉   | 6940/10000 [10:54<08:43,  5.85it/s, est. speed input: 3314.45 toks/s, output: 818.33 toks/s]
Processed prompts:  69%|██████▉   | 6945/10000 [10:54<06:55,  7.36it/s, est. speed input: 3317.03 toks/s, output: 818.43 toks/s]
Processed prompts:  69%|██████▉   | 6948/10000 [10:55<08:08,  6.25it/s, est. speed input: 3316.67 toks/s, output: 817.97 toks/s]
Processed prompts:  70%|██████▉   | 6952/10000 [10:56<07:27,  6.81it/s, est. speed input: 3319.66 toks/s, output: 817.92 toks/s]
Processed prompts:  70%|██████▉   | 6955/10000 [10:56<07:40,  6.61it/s, est. speed input: 3319.37 toks/s, output: 817.69 toks/s]
Processed prompts:  70%|██████▉   | 6957/10000 [10:57<09:05,  5.57it/s, est. speed input: 3318.49 toks/s, output: 817.24 toks/s]
Processed prompts:  70%|██████▉   | 6960/10000 [10:57<08:41,  5.83it/s, est. speed input: 3318.98 toks/s, output: 817.04 toks/s]
Processed prompts:  70%|██████▉   | 6966/10000 [10:58<06:19,  7.99it/s, est. speed input: 3323.33 toks/s, output: 817.20 toks/s]
Processed prompts:  70%|██████▉   | 6967/10000 [10:58<09:42,  5.21it/s, est. speed input: 3320.96 toks/s, output: 816.36 toks/s]
Processed prompts:  70%|██████▉   | 6970/10000 [10:59<08:52,  5.69it/s, est. speed input: 3322.21 toks/s, output: 816.27 toks/s]
Processed prompts:  70%|██████▉   | 6973/10000 [10:59<08:14,  6.12it/s, est. speed input: 3323.81 toks/s, output: 816.17 toks/s]
Processed prompts:  70%|██████▉   | 6977/10000 [11:00<07:39,  6.58it/s, est. speed input: 3325.50 toks/s, output: 816.01 toks/s]
Processed prompts:  70%|██████▉   | 6979/10000 [11:00<08:48,  5.72it/s, est. speed input: 3325.24 toks/s, output: 815.59 toks/s]
Processed prompts:  70%|██████▉   | 6980/10000 [11:01<10:21,  4.86it/s, est. speed input: 3324.86 toks/s, output: 815.22 toks/s]
Processed prompts:  70%|██████▉   | 6982/10000 [11:01<09:40,  5.20it/s, est. speed input: 3326.46 toks/s, output: 815.09 toks/s]
Processed prompts:  70%|██████▉   | 6985/10000 [11:01<08:14,  6.10it/s, est. speed input: 3329.28 toks/s, output: 815.08 toks/s]
Processed prompts:  70%|██████▉   | 6988/10000 [11:02<08:01,  6.26it/s, est. speed input: 3331.07 toks/s, output: 814.93 toks/s]
Processed prompts:  70%|██████▉   | 6993/10000 [11:02<06:26,  7.77it/s, est. speed input: 3334.21 toks/s, output: 815.00 toks/s]
Processed prompts:  70%|██████▉   | 6995/10000 [11:03<08:10,  6.12it/s, est. speed input: 3333.80 toks/s, output: 814.57 toks/s]
Processed prompts:  70%|███████   | 7000/10000 [11:03<06:45,  7.39it/s, est. speed input: 3335.37 toks/s, output: 814.62 toks/s]
Processed prompts:  70%|███████   | 7001/10000 [11:04<10:13,  4.89it/s, est. speed input: 3332.61 toks/s, output: 813.83 toks/s]
Processed prompts:  70%|███████   | 7004/10000 [11:04<08:37,  5.79it/s, est. speed input: 3334.88 toks/s, output: 813.83 toks/s]
Processed prompts:  70%|███████   | 7007/10000 [11:05<08:26,  5.91it/s, est. speed input: 3335.98 toks/s, output: 813.63 toks/s]
Processed prompts:  70%|███████   | 7011/10000 [11:05<07:22,  6.76it/s, est. speed input: 3338.14 toks/s, output: 813.60 toks/s]
Processed prompts:  70%|███████   | 7013/10000 [11:06<08:54,  5.59it/s, est. speed input: 3336.94 toks/s, output: 813.13 toks/s]
Processed prompts:  70%|███████   | 7017/10000 [11:06<07:15,  6.86it/s, est. speed input: 3340.16 toks/s, output: 813.22 toks/s]
Processed prompts:  70%|███████   | 7019/10000 [11:07<08:54,  5.58it/s, est. speed input: 3338.76 toks/s, output: 812.73 toks/s]
Processed prompts:  70%|███████   | 7021/10000 [11:07<08:28,  5.86it/s, est. speed input: 3339.68 toks/s, output: 812.67 toks/s]
Processed prompts:  70%|███████   | 7025/10000 [11:07<06:27,  7.69it/s, est. speed input: 3342.99 toks/s, output: 812.86 toks/s]
Processed prompts:  70%|███████   | 7027/10000 [11:08<06:36,  7.50it/s, est. speed input: 3344.49 toks/s, output: 812.78 toks/s]
Processed prompts:  70%|███████   | 7029/10000 [11:08<06:22,  7.76it/s, est. speed input: 3344.64 toks/s, output: 812.76 toks/s]
Processed prompts:  70%|███████   | 7030/10000 [11:08<07:06,  6.97it/s, est. speed input: 3344.19 toks/s, output: 812.60 toks/s]
Processed prompts:  70%|███████   | 7031/10000 [11:08<07:34,  6.53it/s, est. speed input: 3344.08 toks/s, output: 812.50 toks/s]
Processed prompts:  70%|███████   | 7035/10000 [11:09<05:08,  9.62it/s, est. speed input: 3347.11 toks/s, output: 812.76 toks/s]
Processed prompts:  70%|███████   | 7038/10000 [11:09<05:01,  9.84it/s, est. speed input: 3350.01 toks/s, output: 812.77 toks/s]
Processed prompts:  70%|███████   | 7040/10000 [11:09<05:22,  9.17it/s, est. speed input: 3351.14 toks/s, output: 812.70 toks/s]
Processed prompts:  70%|███████   | 7049/10000 [11:09<02:49, 17.44it/s, est. speed input: 3358.81 toks/s, output: 813.51 toks/s]
Processed prompts:  71%|███████   | 7053/10000 [11:10<03:33, 13.78it/s, est. speed input: 3362.10 toks/s, output: 813.47 toks/s]
Processed prompts:  71%|███████   | 7055/10000 [11:10<04:08, 11.85it/s, est. speed input: 3363.57 toks/s, output: 813.36 toks/s]
Processed prompts:  71%|███████   | 7058/10000 [11:10<04:02, 12.14it/s, est. speed input: 3365.31 toks/s, output: 813.44 toks/s]
Processed prompts:  71%|███████   | 7061/10000 [11:11<04:08, 11.83it/s, est. speed input: 3368.07 toks/s, output: 813.47 toks/s]
Processed prompts:  71%|███████   | 7065/10000 [11:11<03:51, 12.69it/s, est. speed input: 3370.55 toks/s, output: 813.65 toks/s]
Processed prompts:  71%|███████   | 7068/10000 [11:11<04:04, 11.98it/s, est. speed input: 3373.16 toks/s, output: 813.68 toks/s]
Processed prompts:  71%|███████   | 7071/10000 [11:11<04:08, 11.79it/s, est. speed input: 3374.80 toks/s, output: 813.76 toks/s]
Processed prompts:  71%|███████   | 7076/10000 [11:12<03:30, 13.89it/s, est. speed input: 3378.54 toks/s, output: 814.06 toks/s]
Processed prompts:  71%|███████   | 7079/10000 [11:12<03:56, 12.34it/s, est. speed input: 3379.29 toks/s, output: 814.05 toks/s]
Processed prompts:  71%|███████   | 7081/10000 [11:12<04:25, 10.98it/s, est. speed input: 3379.55 toks/s, output: 813.97 toks/s]
Processed prompts:  71%|███████   | 7086/10000 [11:13<03:30, 13.84it/s, est. speed input: 3384.30 toks/s, output: 814.30 toks/s]
Processed prompts:  71%|███████   | 7090/10000 [11:13<03:38, 13.30it/s, est. speed input: 3387.22 toks/s, output: 814.40 toks/s]
Processed prompts:  71%|███████   | 7092/10000 [11:13<04:13, 11.45it/s, est. speed input: 3387.63 toks/s, output: 814.35 toks/s]
Processed prompts:  71%|███████   | 7095/10000 [11:13<03:54, 12.38it/s, est. speed input: 3390.29 toks/s, output: 814.48 toks/s]
Processed prompts:  71%|███████   | 7098/10000 [11:13<03:22, 14.31it/s, est. speed input: 3392.56 toks/s, output: 814.72 toks/s]
Processed prompts:  71%|███████   | 7100/10000 [11:14<03:11, 15.16it/s, est. speed input: 3393.23 toks/s, output: 814.87 toks/s]
Processed prompts:  71%|███████   | 7104/10000 [11:14<02:44, 17.56it/s, est. speed input: 3396.77 toks/s, output: 815.22 toks/s]
Processed prompts:  71%|███████   | 7110/10000 [11:14<02:12, 21.86it/s, est. speed input: 3402.54 toks/s, output: 815.75 toks/s]
Processed prompts:  71%|███████   | 7120/10000 [11:14<01:28, 32.38it/s, est. speed input: 3411.92 toks/s, output: 816.77 toks/s]
Processed prompts:  71%|███████   | 7124/10000 [11:14<01:36, 29.95it/s, est. speed input: 3415.71 toks/s, output: 817.04 toks/s]
Processed prompts:  71%|███████▏  | 7128/10000 [11:14<01:44, 27.49it/s, est. speed input: 3419.65 toks/s, output: 817.29 toks/s]
Processed prompts:  71%|███████▏  | 7133/10000 [11:15<01:46, 26.83it/s, est. speed input: 3423.47 toks/s, output: 817.65 toks/s]
Processed prompts:  71%|███████▏  | 7138/10000 [11:15<01:40, 28.47it/s, est. speed input: 3427.96 toks/s, output: 818.16 toks/s]
Processed prompts:  71%|███████▏  | 7144/10000 [11:15<01:23, 34.20it/s, est. speed input: 3432.95 toks/s, output: 818.82 toks/s]
Processed prompts:  72%|███████▏  | 7152/10000 [11:15<01:09, 41.24it/s, est. speed input: 3440.34 toks/s, output: 819.64 toks/s]
Processed prompts:  72%|███████▏  | 7157/10000 [11:15<01:30, 31.37it/s, est. speed input: 3442.46 toks/s, output: 819.91 toks/s]
Processed prompts:  72%|███████▏  | 7163/10000 [11:15<01:27, 32.56it/s, est. speed input: 3447.93 toks/s, output: 820.41 toks/s]
Processed prompts:  72%|███████▏  | 7168/10000 [11:16<01:34, 30.00it/s, est. speed input: 3451.33 toks/s, output: 820.77 toks/s]
Processed prompts:  72%|███████▏  | 7173/10000 [11:16<01:28, 31.79it/s, est. speed input: 3455.21 toks/s, output: 821.15 toks/s]
Processed prompts:  72%|███████▏  | 7177/10000 [11:16<01:36, 29.18it/s, est. speed input: 3459.42 toks/s, output: 821.48 toks/s]
Processed prompts:  72%|███████▏  | 7183/10000 [11:16<01:36, 29.24it/s, est. speed input: 3463.72 toks/s, output: 821.97 toks/s]
Processed prompts:  72%|███████▏  | 7189/10000 [11:16<01:27, 32.24it/s, est. speed input: 3469.95 toks/s, output: 822.53 toks/s]
Processed prompts:  72%|███████▏  | 7193/10000 [11:17<01:42, 27.51it/s, est. speed input: 3473.57 toks/s, output: 822.75 toks/s]
Processed prompts:  72%|███████▏  | 7198/10000 [11:17<01:44, 26.87it/s, est. speed input: 3476.56 toks/s, output: 823.14 toks/s]
Processed prompts:  72%|███████▏  | 7206/10000 [11:17<01:28, 31.73it/s, est. speed input: 3484.56 toks/s, output: 823.90 toks/s]
Processed prompts:  72%|███████▏  | 7214/10000 [11:17<01:23, 33.44it/s, est. speed input: 3493.68 toks/s, output: 824.69 toks/s]
Processed prompts:  72%|███████▏  | 7224/10000 [11:17<01:15, 36.68it/s, est. speed input: 3503.59 toks/s, output: 825.62 toks/s]
Processed prompts:  72%|███████▏  | 7228/10000 [11:18<01:33, 29.64it/s, est. speed input: 3506.73 toks/s, output: 825.74 toks/s]
Processed prompts:  72%|███████▏  | 7235/10000 [11:18<01:24, 32.60it/s, est. speed input: 3508.97 toks/s, output: 826.23 toks/s]
Processed prompts:  72%|███████▏  | 7240/10000 [11:18<01:27, 31.37it/s, est. speed input: 3511.69 toks/s, output: 826.59 toks/s]
Processed prompts:  72%|███████▏  | 7246/10000 [11:18<01:29, 30.89it/s, est. speed input: 3517.48 toks/s, output: 827.04 toks/s]
Processed prompts:  73%|███████▎  | 7253/10000 [11:18<01:23, 32.93it/s, est. speed input: 3519.98 toks/s, output: 827.59 toks/s]
Processed prompts:  73%|███████▎  | 7264/10000 [11:19<01:13, 37.13it/s, est. speed input: 3528.93 toks/s, output: 828.59 toks/s]
Processed prompts:  73%|███████▎  | 7268/10000 [11:19<01:24, 32.21it/s, est. speed input: 3530.34 toks/s, output: 828.80 toks/s]
Processed prompts:  73%|███████▎  | 7282/10000 [11:19<00:58, 46.31it/s, est. speed input: 3536.39 toks/s, output: 830.15 toks/s]
Processed prompts:  73%|███████▎  | 7287/10000 [11:19<01:09, 39.32it/s, est. speed input: 3537.80 toks/s, output: 830.44 toks/s]
Processed prompts:  73%|███████▎  | 7292/10000 [11:19<01:06, 40.98it/s, est. speed input: 3539.17 toks/s, output: 830.85 toks/s]
Processed prompts:  73%|███████▎  | 7297/10000 [11:19<01:03, 42.54it/s, est. speed input: 3541.12 toks/s, output: 831.23 toks/s]
Processed prompts:  73%|███████▎  | 7303/10000 [11:20<01:04, 41.76it/s, est. speed input: 3542.91 toks/s, output: 831.63 toks/s]
Processed prompts:  73%|███████▎  | 7308/10000 [11:20<01:06, 40.60it/s, est. speed input: 3545.08 toks/s, output: 832.03 toks/s]
Processed prompts:  73%|███████▎  | 7317/10000 [11:20<00:54, 49.44it/s, est. speed input: 3549.88 toks/s, output: 832.90 toks/s]
Processed prompts:  73%|███████▎  | 7323/10000 [11:20<01:12, 36.90it/s, est. speed input: 3551.79 toks/s, output: 833.26 toks/s]
Processed prompts:  73%|███████▎  | 7331/10000 [11:20<01:10, 38.02it/s, est. speed input: 3554.51 toks/s, output: 833.95 toks/s]
Processed prompts:  73%|███████▎  | 7336/10000 [11:20<01:06, 40.09it/s, est. speed input: 3555.55 toks/s, output: 834.35 toks/s]
Processed prompts:  73%|███████▎  | 7341/10000 [11:21<01:14, 35.60it/s, est. speed input: 3556.88 toks/s, output: 834.70 toks/s]
Processed prompts:  73%|███████▎  | 7348/10000 [11:21<01:02, 42.40it/s, est. speed input: 3560.10 toks/s, output: 835.46 toks/s]
Processed prompts:  74%|███████▎  | 7353/10000 [11:21<01:17, 34.27it/s, est. speed input: 3562.70 toks/s, output: 835.86 toks/s]
Processed prompts:  74%|███████▎  | 7357/10000 [11:21<01:26, 30.44it/s, est. speed input: 3564.42 toks/s, output: 836.10 toks/s]
Processed prompts:  74%|███████▎  | 7364/10000 [11:21<01:17, 33.94it/s, est. speed input: 3565.72 toks/s, output: 836.64 toks/s]
Processed prompts:  74%|███████▎  | 7369/10000 [11:21<01:23, 31.70it/s, est. speed input: 3566.74 toks/s, output: 836.97 toks/s]
Processed prompts:  74%|███████▍  | 7382/10000 [11:22<01:06, 39.43it/s, est. speed input: 3569.45 toks/s, output: 838.03 toks/s]
Processed prompts:  74%|███████▍  | 7389/10000 [11:22<00:58, 44.70it/s, est. speed input: 3571.05 toks/s, output: 838.65 toks/s]
Processed prompts:  74%|███████▍  | 7394/10000 [11:22<01:02, 41.63it/s, est. speed input: 3571.77 toks/s, output: 838.95 toks/s]
Processed prompts:  74%|███████▍  | 7400/10000 [11:22<01:07, 38.38it/s, est. speed input: 3572.66 toks/s, output: 839.32 toks/s]
Processed prompts:  74%|███████▍  | 7405/10000 [11:22<01:26, 30.05it/s, est. speed input: 3572.76 toks/s, output: 839.51 toks/s]
Processed prompts:  74%|███████▍  | 7410/10000 [11:23<01:22, 31.33it/s, est. speed input: 3573.62 toks/s, output: 839.89 toks/s]
Processed prompts:  74%|███████▍  | 7415/10000 [11:23<01:30, 28.65it/s, est. speed input: 3574.07 toks/s, output: 840.17 toks/s]
Processed prompts:  74%|███████▍  | 7419/10000 [11:23<01:26, 29.86it/s, est. speed input: 3574.69 toks/s, output: 840.42 toks/s]
Processed prompts:  74%|███████▍  | 7424/10000 [11:23<01:20, 31.97it/s, est. speed input: 3575.60 toks/s, output: 840.70 toks/s]
Processed prompts:  74%|███████▍  | 7429/10000 [11:23<01:12, 35.65it/s, est. speed input: 3576.58 toks/s, output: 841.10 toks/s]
Processed prompts:  74%|███████▍  | 7434/10000 [11:23<01:10, 36.35it/s, est. speed input: 3577.49 toks/s, output: 841.43 toks/s]
Processed prompts:  74%|███████▍  | 7438/10000 [11:23<01:27, 29.37it/s, est. speed input: 3577.58 toks/s, output: 841.64 toks/s]
Processed prompts:  74%|███████▍  | 7443/10000 [11:24<01:32, 27.56it/s, est. speed input: 3578.09 toks/s, output: 841.92 toks/s]
Processed prompts:  74%|███████▍  | 7449/10000 [11:24<01:29, 28.52it/s, est. speed input: 3578.88 toks/s, output: 842.28 toks/s]
Processed prompts:  75%|███████▍  | 7453/10000 [11:24<01:36, 26.39it/s, est. speed input: 3579.22 toks/s, output: 842.48 toks/s]
Processed prompts:  75%|███████▍  | 7459/10000 [11:24<02:03, 20.54it/s, est. speed input: 3578.88 toks/s, output: 842.61 toks/s]
Processed prompts:  75%|███████▍  | 7466/10000 [11:25<01:36, 26.30it/s, est. speed input: 3580.29 toks/s, output: 843.19 toks/s]
Processed prompts:  75%|███████▍  | 7470/10000 [11:25<01:36, 26.14it/s, est. speed input: 3580.75 toks/s, output: 843.41 toks/s]
Processed prompts:  75%|███████▍  | 7475/10000 [11:25<01:31, 27.69it/s, est. speed input: 3581.51 toks/s, output: 843.74 toks/s]
Processed prompts:  75%|███████▍  | 7482/10000 [11:25<01:21, 30.81it/s, est. speed input: 3582.70 toks/s, output: 844.23 toks/s]
Processed prompts:  75%|███████▍  | 7488/10000 [11:25<01:09, 36.18it/s, est. speed input: 3584.04 toks/s, output: 844.73 toks/s]
Processed prompts:  75%|███████▍  | 7493/10000 [11:25<01:08, 36.73it/s, est. speed input: 3584.92 toks/s, output: 845.10 toks/s]
Processed prompts:  75%|███████▍  | 7498/10000 [11:26<01:17, 32.45it/s, est. speed input: 3585.40 toks/s, output: 845.39 toks/s]
Processed prompts:  75%|███████▌  | 7507/10000 [11:26<01:05, 38.00it/s, est. speed input: 3587.24 toks/s, output: 846.05 toks/s]
Processed prompts:  75%|███████▌  | 7511/10000 [11:26<01:08, 36.27it/s, est. speed input: 3587.87 toks/s, output: 846.30 toks/s]
Processed prompts:  75%|███████▌  | 7515/10000 [11:26<01:07, 36.92it/s, est. speed input: 3588.59 toks/s, output: 846.58 toks/s]
Processed prompts:  75%|███████▌  | 7519/10000 [11:26<01:26, 28.69it/s, est. speed input: 3588.60 toks/s, output: 846.70 toks/s]
Processed prompts:  75%|███████▌  | 7527/10000 [11:26<01:15, 32.96it/s, est. speed input: 3590.08 toks/s, output: 847.24 toks/s]
Processed prompts:  75%|███████▌  | 7534/10000 [11:27<01:09, 35.26it/s, est. speed input: 3591.35 toks/s, output: 847.73 toks/s]
Processed prompts:  75%|███████▌  | 7538/10000 [11:27<01:10, 34.71it/s, est. speed input: 3591.98 toks/s, output: 848.00 toks/s]
Processed prompts:  75%|███████▌  | 7544/10000 [11:27<01:12, 33.65it/s, est. speed input: 3592.84 toks/s, output: 848.38 toks/s]
Processed prompts:  75%|███████▌  | 7549/10000 [11:27<01:06, 36.78it/s, est. speed input: 3593.84 toks/s, output: 848.74 toks/s]
Processed prompts:  76%|███████▌  | 7553/10000 [11:27<01:08, 35.85it/s, est. speed input: 3594.44 toks/s, output: 848.99 toks/s]
Processed prompts:  76%|███████▌  | 7559/10000 [11:27<00:59, 41.22it/s, est. speed input: 3595.76 toks/s, output: 849.49 toks/s]
Processed prompts:  76%|███████▌  | 7564/10000 [11:27<01:00, 40.39it/s, est. speed input: 3596.61 toks/s, output: 849.91 toks/s]
Processed prompts:  76%|███████▌  | 7572/10000 [11:27<01:01, 39.22it/s, est. speed input: 3597.92 toks/s, output: 850.53 toks/s]
Processed prompts:  76%|███████▌  | 7577/10000 [11:28<01:00, 39.74it/s, est. speed input: 3598.83 toks/s, output: 850.91 toks/s]
Processed prompts:  76%|███████▌  | 7582/10000 [11:28<00:58, 41.62it/s, est. speed input: 3599.84 toks/s, output: 851.37 toks/s]
Processed prompts:  76%|███████▌  | 7592/10000 [11:28<00:59, 40.73it/s, est. speed input: 3601.62 toks/s, output: 852.14 toks/s]
Processed prompts:  76%|███████▌  | 7597/10000 [11:28<00:58, 40.95it/s, est. speed input: 3602.56 toks/s, output: 852.47 toks/s]
Processed prompts:  76%|███████▌  | 7606/10000 [11:28<00:56, 42.52it/s, est. speed input: 3604.32 toks/s, output: 853.16 toks/s]
Processed prompts:  76%|███████▌  | 7611/10000 [11:29<01:10, 33.90it/s, est. speed input: 3604.65 toks/s, output: 853.43 toks/s]
Processed prompts:  76%|███████▌  | 7617/10000 [11:29<01:11, 33.49it/s, est. speed input: 3605.54 toks/s, output: 853.81 toks/s]
Processed prompts:  76%|███████▌  | 7623/10000 [11:29<01:11, 33.35it/s, est. speed input: 3606.46 toks/s, output: 854.22 toks/s]
Processed prompts:  76%|███████▋  | 7629/10000 [11:29<01:12, 32.89it/s, est. speed input: 3607.37 toks/s, output: 854.64 toks/s]
Processed prompts:  76%|███████▋  | 7639/10000 [11:29<01:01, 38.42it/s, est. speed input: 3609.44 toks/s, output: 855.38 toks/s]
Processed prompts:  76%|███████▋  | 7643/10000 [11:29<01:03, 37.27it/s, est. speed input: 3610.05 toks/s, output: 855.69 toks/s]
Processed prompts:  76%|███████▋  | 7647/10000 [11:30<01:02, 37.57it/s, est. speed input: 3610.73 toks/s, output: 855.98 toks/s]
Processed prompts:  77%|███████▋  | 7651/10000 [11:30<01:02, 37.70it/s, est. speed input: 3611.39 toks/s, output: 856.25 toks/s]
Processed prompts:  77%|███████▋  | 7655/10000 [11:30<01:01, 37.89it/s, est. speed input: 3612.05 toks/s, output: 856.57 toks/s]
Processed prompts:  77%|███████▋  | 7661/10000 [11:30<01:06, 34.93it/s, est. speed input: 3612.83 toks/s, output: 856.87 toks/s]
Processed prompts:  77%|███████▋  | 7667/10000 [11:30<01:07, 34.69it/s, est. speed input: 3613.77 toks/s, output: 857.28 toks/s]
Processed prompts:  77%|███████▋  | 7671/10000 [11:30<01:16, 30.54it/s, est. speed input: 3614.05 toks/s, output: 857.42 toks/s]
Processed prompts:  77%|███████▋  | 7681/10000 [11:30<01:01, 37.44it/s, est. speed input: 3616.04 toks/s, output: 858.17 toks/s]
Processed prompts:  77%|███████▋  | 7685/10000 [11:31<01:03, 36.50it/s, est. speed input: 3616.61 toks/s, output: 858.44 toks/s]
Processed prompts:  77%|███████▋  | 7689/10000 [11:31<01:02, 36.96it/s, est. speed input: 3617.30 toks/s, output: 858.79 toks/s]
Processed prompts:  77%|███████▋  | 7698/10000 [11:31<00:57, 40.05it/s, est. speed input: 3619.02 toks/s, output: 859.47 toks/s]
Processed prompts:  77%|███████▋  | 7702/10000 [11:31<01:01, 37.30it/s, est. speed input: 3619.55 toks/s, output: 859.69 toks/s]
Processed prompts:  77%|███████▋  | 7706/10000 [11:31<01:01, 37.55it/s, est. speed input: 3620.21 toks/s, output: 859.98 toks/s]
Processed prompts:  77%|███████▋  | 7711/10000 [11:31<01:12, 31.64it/s, est. speed input: 3620.59 toks/s, output: 860.16 toks/s]
Processed prompts:  77%|███████▋  | 7716/10000 [11:31<01:09, 32.77it/s, est. speed input: 3621.43 toks/s, output: 860.48 toks/s]
Processed prompts:  77%|███████▋  | 7720/10000 [11:32<01:14, 30.43it/s, est. speed input: 3621.79 toks/s, output: 860.72 toks/s]
Processed prompts:  77%|███████▋  | 7726/10000 [11:32<01:13, 30.89it/s, est. speed input: 3622.62 toks/s, output: 861.09 toks/s]
Processed prompts:  77%|███████▋  | 7734/10000 [11:32<01:07, 33.35it/s, est. speed input: 3623.91 toks/s, output: 861.55 toks/s]
Processed prompts:  77%|███████▋  | 7742/10000 [11:32<01:14, 30.19it/s, est. speed input: 3624.78 toks/s, output: 861.97 toks/s]
Processed prompts:  77%|███████▋  | 7746/10000 [11:32<01:12, 30.95it/s, est. speed input: 3625.39 toks/s, output: 862.24 toks/s]
Processed prompts:  78%|███████▊  | 7755/10000 [11:33<01:06, 33.87it/s, est. speed input: 3626.98 toks/s, output: 862.96 toks/s]
Processed prompts:  78%|███████▊  | 7763/10000 [11:33<00:58, 38.15it/s, est. speed input: 3628.62 toks/s, output: 863.59 toks/s]
Processed prompts:  78%|███████▊  | 7768/10000 [11:33<00:58, 38.28it/s, est. speed input: 3629.46 toks/s, output: 863.86 toks/s]
Processed prompts:  78%|███████▊  | 7772/10000 [11:33<01:00, 37.09it/s, est. speed input: 3630.06 toks/s, output: 864.09 toks/s]
Processed prompts:  78%|███████▊  | 7776/10000 [11:33<00:59, 37.43it/s, est. speed input: 3630.77 toks/s, output: 864.34 toks/s]
Processed prompts:  78%|███████▊  | 7783/10000 [11:33<00:49, 44.70it/s, est. speed input: 3632.35 toks/s, output: 864.88 toks/s]
Processed prompts:  78%|███████▊  | 7789/10000 [11:33<00:51, 43.10it/s, est. speed input: 3633.43 toks/s, output: 865.31 toks/s]
Processed prompts:  78%|███████▊  | 7794/10000 [11:34<00:53, 41.46it/s, est. speed input: 3634.25 toks/s, output: 865.63 toks/s]
Processed prompts:  78%|███████▊  | 7799/10000 [11:34<01:02, 34.98it/s, est. speed input: 3634.73 toks/s, output: 865.92 toks/s]
Processed prompts:  78%|███████▊  | 7807/10000 [11:34<00:56, 38.73it/s, est. speed input: 3636.26 toks/s, output: 866.47 toks/s]
Processed prompts:  78%|███████▊  | 7812/10000 [11:34<01:05, 33.54it/s, est. speed input: 3636.72 toks/s, output: 866.70 toks/s]
Processed prompts:  78%|███████▊  | 7816/10000 [11:34<01:14, 29.39it/s, est. speed input: 3636.91 toks/s, output: 866.89 toks/s]
Processed prompts:  78%|███████▊  | 7826/10000 [11:35<00:59, 36.53it/s, est. speed input: 3638.99 toks/s, output: 867.72 toks/s]
Processed prompts:  78%|███████▊  | 7831/10000 [11:35<00:55, 38.90it/s, est. speed input: 3639.95 toks/s, output: 868.09 toks/s]
Processed prompts:  78%|███████▊  | 7838/10000 [11:35<00:49, 43.57it/s, est. speed input: 3641.39 toks/s, output: 868.59 toks/s]
Processed prompts:  78%|███████▊  | 7843/10000 [11:35<01:03, 33.85it/s, est. speed input: 3641.62 toks/s, output: 868.81 toks/s]
Processed prompts:  79%|███████▊  | 7852/10000 [11:35<00:57, 37.62it/s, est. speed input: 3643.30 toks/s, output: 869.50 toks/s]
Processed prompts:  79%|███████▊  | 7858/10000 [11:35<00:52, 41.17it/s, est. speed input: 3644.55 toks/s, output: 869.95 toks/s]
Processed prompts:  79%|███████▊  | 7863/10000 [11:36<01:00, 35.34it/s, est. speed input: 3645.02 toks/s, output: 870.19 toks/s]
Processed prompts:  79%|███████▊  | 7867/10000 [11:36<01:07, 31.80it/s, est. speed input: 3645.32 toks/s, output: 870.36 toks/s]
Processed prompts:  79%|███████▊  | 7871/10000 [11:36<01:47, 19.89it/s, est. speed input: 3644.31 toks/s, output: 870.22 toks/s]
Processed prompts:  79%|███████▉  | 7876/10000 [11:36<01:39, 21.42it/s, est. speed input: 3644.84 toks/s, output: 870.52 toks/s]
Processed prompts:  79%|███████▉  | 7879/10000 [11:37<02:03, 17.22it/s, est. speed input: 3644.15 toks/s, output: 870.43 toks/s]
Processed prompts:  79%|███████▉  | 7882/10000 [11:37<02:38, 13.39it/s, est. speed input: 3642.97 toks/s, output: 870.25 toks/s]
Processed prompts:  79%|███████▉  | 7885/10000 [11:37<02:32, 13.90it/s, est. speed input: 3642.93 toks/s, output: 870.31 toks/s]
Processed prompts:  79%|███████▉  | 7888/10000 [11:37<02:33, 13.79it/s, est. speed input: 3642.74 toks/s, output: 870.35 toks/s]
Processed prompts:  79%|███████▉  | 7890/10000 [11:38<02:50, 12.39it/s, est. speed input: 3642.14 toks/s, output: 870.32 toks/s]
Processed prompts:  79%|███████▉  | 7894/10000 [11:38<02:18, 15.16it/s, est. speed input: 3642.53 toks/s, output: 870.50 toks/s]
Processed prompts:  79%|███████▉  | 7897/10000 [11:38<02:35, 13.53it/s, est. speed input: 3641.98 toks/s, output: 870.47 toks/s]
Processed prompts:  79%|███████▉  | 7901/10000 [11:38<02:22, 14.74it/s, est. speed input: 3642.01 toks/s, output: 870.58 toks/s]
Processed prompts:  79%|███████▉  | 7903/10000 [11:39<02:45, 12.65it/s, est. speed input: 3641.32 toks/s, output: 870.50 toks/s]
Processed prompts:  79%|███████▉  | 7910/10000 [11:39<01:46, 19.65it/s, est. speed input: 3642.73 toks/s, output: 870.97 toks/s]
Processed prompts:  79%|███████▉  | 7913/10000 [11:39<02:21, 14.71it/s, est. speed input: 3641.72 toks/s, output: 870.82 toks/s]
Processed prompts:  79%|███████▉  | 7918/10000 [11:39<02:05, 16.60it/s, est. speed input: 3642.02 toks/s, output: 871.03 toks/s]
Processed prompts:  79%|███████▉  | 7921/10000 [11:40<02:21, 14.65it/s, est. speed input: 3641.44 toks/s, output: 870.96 toks/s]
Processed prompts:  79%|███████▉  | 7923/10000 [11:40<02:39, 13.05it/s, est. speed input: 3640.87 toks/s, output: 870.89 toks/s]
Processed prompts:  79%|███████▉  | 7929/10000 [11:40<01:57, 17.66it/s, est. speed input: 3641.69 toks/s, output: 871.30 toks/s]
Processed prompts:  79%|███████▉  | 7935/10000 [11:40<01:52, 18.43it/s, est. speed input: 3641.99 toks/s, output: 871.53 toks/s]
Processed prompts:  79%|███████▉  | 7937/10000 [11:41<02:24, 14.29it/s, est. speed input: 3640.97 toks/s, output: 871.36 toks/s]
Processed prompts:  79%|███████▉  | 7942/10000 [11:41<01:59, 17.15it/s, est. speed input: 3641.48 toks/s, output: 871.63 toks/s]
Processed prompts:  79%|███████▉  | 7949/10000 [11:41<01:44, 19.63it/s, est. speed input: 3642.12 toks/s, output: 871.97 toks/s]
Processed prompts:  80%|███████▉  | 7953/10000 [11:42<02:03, 16.60it/s, est. speed input: 3641.49 toks/s, output: 871.93 toks/s]
Processed prompts:  80%|███████▉  | 7956/10000 [11:42<02:12, 15.40it/s, est. speed input: 3641.11 toks/s, output: 871.93 toks/s]
Processed prompts:  80%|███████▉  | 7960/10000 [11:42<02:06, 16.12it/s, est. speed input: 3641.19 toks/s, output: 872.08 toks/s]
Processed prompts:  80%|███████▉  | 7967/10000 [11:42<01:43, 19.69it/s, est. speed input: 3642.01 toks/s, output: 872.40 toks/s]
Processed prompts:  80%|███████▉  | 7971/10000 [11:43<02:01, 16.72it/s, est. speed input: 3641.41 toks/s, output: 872.32 toks/s]
Processed prompts:  80%|███████▉  | 7974/10000 [11:43<02:10, 15.57it/s, est. speed input: 3641.04 toks/s, output: 872.30 toks/s]
Processed prompts:  80%|███████▉  | 7976/10000 [11:43<02:27, 13.68it/s, est. speed input: 3640.44 toks/s, output: 872.18 toks/s]
Processed prompts:  80%|███████▉  | 7979/10000 [11:43<02:22, 14.19it/s, est. speed input: 3640.38 toks/s, output: 872.23 toks/s]
Processed prompts:  80%|███████▉  | 7981/10000 [11:43<02:39, 12.67it/s, est. speed input: 3639.85 toks/s, output: 872.15 toks/s]
Processed prompts:  80%|███████▉  | 7984/10000 [11:44<02:29, 13.49it/s, est. speed input: 3639.78 toks/s, output: 872.22 toks/s]
Processed prompts:  80%|███████▉  | 7988/10000 [11:44<02:15, 14.81it/s, est. speed input: 3639.80 toks/s, output: 872.35 toks/s]
Processed prompts:  80%|███████▉  | 7990/10000 [11:44<02:39, 12.62it/s, est. speed input: 3639.14 toks/s, output: 872.25 toks/s]
Processed prompts:  80%|███████▉  | 7995/10000 [11:44<02:03, 16.18it/s, est. speed input: 3639.66 toks/s, output: 872.48 toks/s]
Processed prompts:  80%|███████▉  | 7998/10000 [11:45<02:21, 14.11it/s, est. speed input: 3639.06 toks/s, output: 872.41 toks/s]
Processed prompts:  80%|████████  | 8002/10000 [11:45<02:12, 15.08it/s, est. speed input: 3639.08 toks/s, output: 872.50 toks/s]
Processed prompts:  80%|████████  | 8004/10000 [11:45<02:34, 12.90it/s, est. speed input: 3638.40 toks/s, output: 872.40 toks/s]
Processed prompts:  80%|████████  | 8006/10000 [11:45<02:35, 12.84it/s, est. speed input: 3638.19 toks/s, output: 872.41 toks/s]
Processed prompts:  80%|████████  | 8009/10000 [11:45<02:26, 13.57it/s, est. speed input: 3638.06 toks/s, output: 872.46 toks/s]
Processed prompts:  80%|████████  | 8012/10000 [11:46<02:32, 13.06it/s, est. speed input: 3637.65 toks/s, output: 872.42 toks/s]
Processed prompts:  80%|████████  | 8017/10000 [11:46<02:05, 15.85it/s, est. speed input: 3638.02 toks/s, output: 872.66 toks/s]
Processed prompts:  80%|████████  | 8022/10000 [11:46<02:00, 16.48it/s, est. speed input: 3638.09 toks/s, output: 872.87 toks/s]
Processed prompts:  80%|████████  | 8026/10000 [11:46<02:06, 15.61it/s, est. speed input: 3637.78 toks/s, output: 872.91 toks/s]
Processed prompts:  80%|████████  | 8030/10000 [11:47<02:04, 15.80it/s, est. speed input: 3637.74 toks/s, output: 873.00 toks/s]
Processed prompts:  80%|████████  | 8035/10000 [11:47<01:54, 17.22it/s, est. speed input: 3638.00 toks/s, output: 873.16 toks/s]
Processed prompts:  80%|████████  | 8037/10000 [11:47<02:21, 13.87it/s, est. speed input: 3637.12 toks/s, output: 873.03 toks/s]
Processed prompts:  80%|████████  | 8041/10000 [11:47<02:05, 15.55it/s, est. speed input: 3637.36 toks/s, output: 873.18 toks/s]
Processed prompts:  80%|████████  | 8043/10000 [11:48<02:57, 11.03it/s, est. speed input: 3635.85 toks/s, output: 872.87 toks/s]
Processed prompts:  80%|████████  | 8050/10000 [11:48<02:13, 14.63it/s, est. speed input: 3636.34 toks/s, output: 873.14 toks/s]
Processed prompts:  81%|████████  | 8054/10000 [11:48<02:14, 14.45it/s, est. speed input: 3636.05 toks/s, output: 873.15 toks/s]
Processed prompts:  81%|████████  | 8057/10000 [11:49<02:20, 13.85it/s, est. speed input: 3635.67 toks/s, output: 873.13 toks/s]
Processed prompts:  81%|████████  | 8063/10000 [11:49<01:53, 17.04it/s, est. speed input: 3636.30 toks/s, output: 873.49 toks/s]
Processed prompts:  81%|████████  | 8066/10000 [11:49<02:10, 14.79it/s, est. speed input: 3635.66 toks/s, output: 873.39 toks/s]
Processed prompts:  81%|████████  | 8068/10000 [11:49<02:26, 13.23it/s, est. speed input: 3635.09 toks/s, output: 873.29 toks/s]
Processed prompts:  81%|████████  | 8070/10000 [11:50<02:33, 12.56it/s, est. speed input: 3634.73 toks/s, output: 873.28 toks/s]
Processed prompts:  81%|████████  | 8077/10000 [11:50<01:42, 18.67it/s, est. speed input: 3635.85 toks/s, output: 873.75 toks/s]
Processed prompts:  81%|████████  | 8080/10000 [11:50<02:11, 14.59it/s, est. speed input: 3634.99 toks/s, output: 873.65 toks/s]
Processed prompts:  81%|████████  | 8086/10000 [11:50<01:46, 17.90it/s, est. speed input: 3635.70 toks/s, output: 873.99 toks/s]
Processed prompts:  81%|████████  | 8090/10000 [11:51<01:58, 16.06it/s, est. speed input: 3635.27 toks/s, output: 873.97 toks/s]
Processed prompts:  81%|████████  | 8092/10000 [11:51<02:18, 13.81it/s, est. speed input: 3634.58 toks/s, output: 873.87 toks/s]
Processed prompts:  81%|████████  | 8097/10000 [11:51<01:54, 16.66it/s, est. speed input: 3635.16 toks/s, output: 874.15 toks/s]
Processed prompts:  81%|████████  | 8101/10000 [11:51<02:00, 15.78it/s, est. speed input: 3634.89 toks/s, output: 874.25 toks/s]
Processed prompts:  81%|████████  | 8104/10000 [11:52<02:08, 14.79it/s, est. speed input: 3634.56 toks/s, output: 874.24 toks/s]
Processed prompts:  81%|████████  | 8106/10000 [11:52<02:25, 12.98it/s, est. speed input: 3633.95 toks/s, output: 874.13 toks/s]
Processed prompts:  81%|████████  | 8108/10000 [11:52<02:33, 12.35it/s, est. speed input: 3633.56 toks/s, output: 874.10 toks/s]
Processed prompts:  81%|████████  | 8110/10000 [11:52<02:39, 11.82it/s, est. speed input: 3633.16 toks/s, output: 874.06 toks/s]
Processed prompts:  81%|████████  | 8112/10000 [11:52<02:45, 11.38it/s, est. speed input: 3632.74 toks/s, output: 874.01 toks/s]
Processed prompts:  81%|████████  | 8118/10000 [11:53<01:51, 16.91it/s, est. speed input: 3633.51 toks/s, output: 874.36 toks/s]
Processed prompts:  81%|████████  | 8121/10000 [11:53<02:12, 14.18it/s, est. speed input: 3632.84 toks/s, output: 874.31 toks/s]
Processed prompts:  81%|████████▏ | 8128/10000 [11:53<01:37, 19.19it/s, est. speed input: 3633.85 toks/s, output: 874.71 toks/s]
Processed prompts:  81%|████████▏ | 8131/10000 [11:54<02:05, 14.93it/s, est. speed input: 3632.94 toks/s, output: 874.57 toks/s]
Processed prompts:  81%|████████▏ | 8134/10000 [11:54<01:58, 15.76it/s, est. speed input: 3633.02 toks/s, output: 874.65 toks/s]
Processed prompts:  81%|████████▏ | 8137/10000 [11:54<02:13, 13.91it/s, est. speed input: 3632.45 toks/s, output: 874.58 toks/s]
Processed prompts:  81%|████████▏ | 8142/10000 [11:54<01:55, 16.02it/s, est. speed input: 3632.74 toks/s, output: 874.77 toks/s]
Processed prompts:  81%|████████▏ | 8144/10000 [11:55<02:49, 10.95it/s, est. speed input: 3631.10 toks/s, output: 874.45 toks/s]
Processed prompts:  82%|████████▏ | 8154/10000 [11:55<01:57, 15.70it/s, est. speed input: 3631.95 toks/s, output: 874.94 toks/s]
Processed prompts:  82%|████████▏ | 8160/10000 [11:55<01:43, 17.72it/s, est. speed input: 3632.54 toks/s, output: 875.21 toks/s]
Processed prompts:  82%|████████▏ | 8164/10000 [11:56<01:52, 16.39it/s, est. speed input: 3632.19 toks/s, output: 875.23 toks/s]
Processed prompts:  82%|████████▏ | 8168/10000 [11:56<01:52, 16.32it/s, est. speed input: 3632.12 toks/s, output: 875.33 toks/s]
Processed prompts:  82%|████████▏ | 8171/10000 [11:56<02:00, 15.18it/s, est. speed input: 3631.77 toks/s, output: 875.31 toks/s]
Processed prompts:  82%|████████▏ | 8173/10000 [11:56<02:15, 13.50it/s, est. speed input: 3631.19 toks/s, output: 875.22 toks/s]
Processed prompts:  82%|████████▏ | 8175/10000 [11:57<02:22, 12.82it/s, est. speed input: 3630.82 toks/s, output: 875.19 toks/s]
Processed prompts:  82%|████████▏ | 8183/10000 [11:57<01:30, 20.12it/s, est. speed input: 3632.29 toks/s, output: 875.75 toks/s]
Processed prompts:  82%|████████▏ | 8187/10000 [11:57<01:52, 16.16it/s, est. speed input: 3631.56 toks/s, output: 875.72 toks/s]
Processed prompts:  82%|████████▏ | 8189/10000 [11:57<02:10, 13.86it/s, est. speed input: 3630.90 toks/s, output: 875.63 toks/s]
Processed prompts:  82%|████████▏ | 8193/10000 [11:58<01:56, 15.51it/s, est. speed input: 3631.08 toks/s, output: 875.76 toks/s]
Processed prompts:  82%|████████▏ | 8195/10000 [11:58<02:16, 13.20it/s, est. speed input: 3630.46 toks/s, output: 875.68 toks/s]
Processed prompts:  82%|████████▏ | 8200/10000 [11:58<02:30, 11.93it/s, est. speed input: 3629.52 toks/s, output: 875.58 toks/s]
Processed prompts:  82%|████████▏ | 8209/10000 [11:59<01:45, 16.97it/s, est. speed input: 3630.67 toks/s, output: 876.10 toks/s]
Processed prompts:  82%|████████▏ | 8214/10000 [11:59<01:46, 16.82it/s, est. speed input: 3630.63 toks/s, output: 876.22 toks/s]
Processed prompts:  82%|████████▏ | 8218/10000 [11:59<01:51, 16.02it/s, est. speed input: 3630.34 toks/s, output: 876.25 toks/s]
Processed prompts:  82%|████████▏ | 8221/10000 [11:59<01:58, 14.99it/s, est. speed input: 3629.94 toks/s, output: 876.26 toks/s]
Processed prompts:  82%|████████▏ | 8225/10000 [12:00<02:25, 12.17it/s, est. speed input: 3628.67 toks/s, output: 876.07 toks/s]
Processed prompts:  82%|████████▏ | 8233/10000 [12:00<01:51, 15.87it/s, est. speed input: 3629.45 toks/s, output: 876.45 toks/s]
Processed prompts:  82%|████████▏ | 8235/10000 [12:01<02:10, 13.47it/s, est. speed input: 3628.60 toks/s, output: 876.31 toks/s]
Processed prompts:  82%|████████▏ | 8243/10000 [12:01<01:29, 19.60it/s, est. speed input: 3630.03 toks/s, output: 876.91 toks/s]
Processed prompts:  82%|████████▏ | 8246/10000 [12:01<01:55, 15.25it/s, est. speed input: 3628.99 toks/s, output: 876.73 toks/s]
Processed prompts:  82%|████████▏ | 8248/10000 [12:01<02:03, 14.24it/s, est. speed input: 3628.61 toks/s, output: 876.71 toks/s]
Processed prompts:  83%|████████▎ | 8254/10000 [12:02<01:40, 17.39it/s, est. speed input: 3629.23 toks/s, output: 877.05 toks/s]
Processed prompts:  83%|████████▎ | 8258/10000 [12:02<01:49, 15.87it/s, est. speed input: 3628.82 toks/s, output: 877.08 toks/s]
Processed prompts:  83%|████████▎ | 8263/10000 [12:02<01:42, 17.00it/s, est. speed input: 3629.01 toks/s, output: 877.31 toks/s]
Processed prompts:  83%|████████▎ | 8268/10000 [12:02<01:41, 17.13it/s, est. speed input: 3629.06 toks/s, output: 877.46 toks/s]
Processed prompts:  83%|████████▎ | 8272/10000 [12:03<01:46, 16.18it/s, est. speed input: 3628.76 toks/s, output: 877.50 toks/s]
Processed prompts:  83%|████████▎ | 8274/10000 [12:03<02:03, 13.98it/s, est. speed input: 3628.15 toks/s, output: 877.40 toks/s]
Processed prompts:  83%|████████▎ | 8276/10000 [12:03<02:05, 13.71it/s, est. speed input: 3627.94 toks/s, output: 877.41 toks/s]
Processed prompts:  83%|████████▎ | 8278/10000 [12:03<02:21, 12.19it/s, est. speed input: 3627.37 toks/s, output: 877.34 toks/s]
Processed prompts:  83%|████████▎ | 8281/10000 [12:04<02:49, 10.14it/s, est. speed input: 3626.22 toks/s, output: 877.14 toks/s]
Processed prompts:  83%|████████▎ | 8289/10000 [12:04<01:49, 15.60it/s, est. speed input: 3627.12 toks/s, output: 877.60 toks/s]
Processed prompts:  83%|████████▎ | 8292/10000 [12:04<02:04, 13.76it/s, est. speed input: 3626.45 toks/s, output: 877.50 toks/s]
Processed prompts:  83%|████████▎ | 8294/10000 [12:05<02:16, 12.50it/s, est. speed input: 3625.89 toks/s, output: 877.42 toks/s]
Processed prompts:  83%|████████▎ | 8301/10000 [12:05<01:34, 18.04it/s, est. speed input: 3626.95 toks/s, output: 877.85 toks/s]
Processed prompts:  83%|████████▎ | 8305/10000 [12:05<01:48, 15.58it/s, est. speed input: 3626.36 toks/s, output: 877.83 toks/s]
Processed prompts:  83%|████████▎ | 8307/10000 [12:05<02:24, 11.75it/s, est. speed input: 3625.04 toks/s, output: 877.56 toks/s]
Processed prompts:  83%|████████▎ | 8309/10000 [12:06<02:22, 11.90it/s, est. speed input: 3624.82 toks/s, output: 877.59 toks/s]
Processed prompts:  83%|████████▎ | 8313/10000 [12:06<01:55, 14.60it/s, est. speed input: 3625.19 toks/s, output: 877.79 toks/s]
Processed prompts:  83%|████████▎ | 8321/10000 [12:06<01:27, 19.21it/s, est. speed input: 3626.14 toks/s, output: 878.22 toks/s]
Processed prompts:  83%|████████▎ | 8324/10000 [12:06<01:52, 14.95it/s, est. speed input: 3625.18 toks/s, output: 878.05 toks/s]
Processed prompts:  83%|████████▎ | 8326/10000 [12:07<02:05, 13.35it/s, est. speed input: 3624.62 toks/s, output: 877.95 toks/s]
Processed prompts:  83%|████████▎ | 8331/10000 [12:07<01:42, 16.23it/s, est. speed input: 3625.09 toks/s, output: 878.18 toks/s]
Processed prompts:  83%|████████▎ | 8333/10000 [12:07<02:06, 13.18it/s, est. speed input: 3624.27 toks/s, output: 878.05 toks/s]
Processed prompts:  83%|████████▎ | 8337/10000 [12:07<01:50, 15.04it/s, est. speed input: 3624.49 toks/s, output: 878.24 toks/s]
Processed prompts:  83%|████████▎ | 8340/10000 [12:08<01:56, 14.23it/s, est. speed input: 3624.15 toks/s, output: 878.23 toks/s]
Processed prompts:  83%|████████▎ | 8342/10000 [12:08<02:12, 12.54it/s, est. speed input: 3623.56 toks/s, output: 878.16 toks/s]
Processed prompts:  83%|████████▎ | 8345/10000 [12:08<02:04, 13.30it/s, est. speed input: 3623.45 toks/s, output: 878.20 toks/s]
Processed prompts:  83%|████████▎ | 8347/10000 [12:08<02:19, 11.89it/s, est. speed input: 3622.88 toks/s, output: 878.12 toks/s]
Processed prompts:  83%|████████▎ | 8349/10000 [12:08<02:24, 11.46it/s, est. speed input: 3622.50 toks/s, output: 878.08 toks/s]
Processed prompts:  84%|████████▎ | 8354/10000 [12:09<01:46, 15.49it/s, est. speed input: 3622.95 toks/s, output: 878.32 toks/s]
Processed prompts:  84%|████████▎ | 8357/10000 [12:09<02:00, 13.62it/s, est. speed input: 3622.43 toks/s, output: 878.28 toks/s]
Processed prompts:  84%|████████▎ | 8361/10000 [12:09<01:51, 14.75it/s, est. speed input: 3622.45 toks/s, output: 878.41 toks/s]
Processed prompts:  84%|████████▎ | 8367/10000 [12:09<01:32, 17.69it/s, est. speed input: 3623.07 toks/s, output: 878.70 toks/s]
Processed prompts:  84%|████████▎ | 8371/10000 [12:10<01:41, 16.09it/s, est. speed input: 3622.67 toks/s, output: 878.70 toks/s]
Processed prompts:  84%|████████▍ | 8375/10000 [12:10<01:41, 16.07it/s, est. speed input: 3622.61 toks/s, output: 878.78 toks/s]
Processed prompts:  84%|████████▍ | 8382/10000 [12:10<01:20, 20.15it/s, est. speed input: 3623.52 toks/s, output: 879.19 toks/s]
Processed prompts:  84%|████████▍ | 8388/10000 [12:11<01:25, 18.96it/s, est. speed input: 3623.52 toks/s, output: 879.37 toks/s]
Processed prompts:  84%|████████▍ | 8390/10000 [12:11<01:47, 14.98it/s, est. speed input: 3622.57 toks/s, output: 879.20 toks/s]
Processed prompts:  84%|████████▍ | 8393/10000 [12:11<01:42, 15.73it/s, est. speed input: 3622.66 toks/s, output: 879.29 toks/s]
Processed prompts:  84%|████████▍ | 8397/10000 [12:11<01:41, 15.82it/s, est. speed input: 3622.62 toks/s, output: 879.40 toks/s]
Processed prompts:  84%|████████▍ | 8402/10000 [12:11<01:33, 17.08it/s, est. speed input: 3622.84 toks/s, output: 879.60 toks/s]
Processed prompts:  84%|████████▍ | 8407/10000 [12:12<01:32, 17.26it/s, est. speed input: 3622.89 toks/s, output: 879.74 toks/s]
Processed prompts:  84%|████████▍ | 8411/10000 [12:12<01:38, 16.21it/s, est. speed input: 3622.63 toks/s, output: 879.80 toks/s]
Processed prompts:  84%|████████▍ | 8413/10000 [12:12<01:54, 13.83it/s, est. speed input: 3621.94 toks/s, output: 879.70 toks/s]
Processed prompts:  84%|████████▍ | 8416/10000 [12:12<01:46, 14.87it/s, est. speed input: 3622.02 toks/s, output: 879.81 toks/s]
Processed prompts:  84%|████████▍ | 8418/10000 [12:13<02:03, 12.82it/s, est. speed input: 3621.38 toks/s, output: 879.71 toks/s]
Processed prompts:  84%|████████▍ | 8423/10000 [12:13<01:37, 16.23it/s, est. speed input: 3621.84 toks/s, output: 879.95 toks/s]
Processed prompts:  84%|████████▍ | 8425/10000 [12:13<02:02, 12.85it/s, est. speed input: 3620.96 toks/s, output: 879.75 toks/s]
Processed prompts:  84%|████████▍ | 8429/10000 [12:13<01:45, 14.85it/s, est. speed input: 3621.17 toks/s, output: 879.91 toks/s]
Processed prompts:  84%|████████▍ | 8431/10000 [12:14<02:04, 12.65it/s, est. speed input: 3620.51 toks/s, output: 879.82 toks/s]
Processed prompts:  84%|████████▍ | 8434/10000 [12:14<01:56, 13.39it/s, est. speed input: 3620.41 toks/s, output: 879.86 toks/s]
Processed prompts:  84%|████████▍ | 8438/10000 [12:14<01:47, 14.57it/s, est. speed input: 3620.40 toks/s, output: 879.92 toks/s]
Processed prompts:  84%|████████▍ | 8440/10000 [12:14<02:05, 12.44it/s, est. speed input: 3619.74 toks/s, output: 879.79 toks/s]
Processed prompts:  84%|████████▍ | 8446/10000 [12:15<01:29, 17.33it/s, est. speed input: 3620.61 toks/s, output: 880.14 toks/s]
Processed prompts:  84%|████████▍ | 8449/10000 [12:15<01:46, 14.51it/s, est. speed input: 3619.96 toks/s, output: 880.06 toks/s]
Processed prompts:  85%|████████▍ | 8451/10000 [12:15<02:00, 12.87it/s, est. speed input: 3619.41 toks/s, output: 880.01 toks/s]
Processed prompts:  85%|████████▍ | 8453/10000 [12:15<02:01, 12.77it/s, est. speed input: 3619.22 toks/s, output: 880.02 toks/s]
Processed prompts:  85%|████████▍ | 8459/10000 [12:15<01:30, 16.97it/s, est. speed input: 3619.89 toks/s, output: 880.31 toks/s]
Processed prompts:  85%|████████▍ | 8464/10000 [12:16<01:31, 16.72it/s, est. speed input: 3619.83 toks/s, output: 880.46 toks/s]
Processed prompts:  85%|████████▍ | 8468/10000 [12:16<01:35, 16.01it/s, est. speed input: 3619.64 toks/s, output: 880.52 toks/s]
Processed prompts:  85%|████████▍ | 8472/10000 [12:16<01:35, 16.04it/s, est. speed input: 3619.59 toks/s, output: 880.57 toks/s]
Processed prompts:  85%|████████▍ | 8476/10000 [12:17<01:34, 16.07it/s, est. speed input: 3619.55 toks/s, output: 880.70 toks/s]
Processed prompts:  85%|████████▍ | 8479/10000 [12:17<01:42, 14.88it/s, est. speed input: 3619.19 toks/s, output: 880.72 toks/s]
Processed prompts:  85%|████████▍ | 8484/10000 [12:17<01:30, 16.82it/s, est. speed input: 3619.51 toks/s, output: 880.95 toks/s]
Processed prompts:  85%|████████▍ | 8488/10000 [12:17<01:35, 15.87it/s, est. speed input: 3619.23 toks/s, output: 880.99 toks/s]
Processed prompts:  85%|████████▍ | 8492/10000 [12:18<01:34, 15.90it/s, est. speed input: 3619.12 toks/s, output: 881.07 toks/s]
Processed prompts:  85%|████████▍ | 8494/10000 [12:18<01:50, 13.62it/s, est. speed input: 3618.48 toks/s, output: 880.98 toks/s]
Processed prompts:  85%|████████▍ | 8498/10000 [12:18<01:37, 15.38it/s, est. speed input: 3618.71 toks/s, output: 881.10 toks/s]
Processed prompts:  85%|████████▌ | 8501/10000 [12:18<01:44, 14.31it/s, est. speed input: 3618.35 toks/s, output: 881.11 toks/s]
Processed prompts:  85%|████████▌ | 8504/10000 [12:19<01:57, 12.73it/s, est. speed input: 3617.72 toks/s, output: 881.03 toks/s]
Processed prompts:  85%|████████▌ | 8507/10000 [12:19<02:07, 11.75it/s, est. speed input: 3617.07 toks/s, output: 880.94 toks/s]
Processed prompts:  85%|████████▌ | 8511/10000 [12:19<02:02, 12.20it/s, est. speed input: 3616.74 toks/s, output: 880.92 toks/s]
Processed prompts:  85%|████████▌ | 8513/10000 [12:19<02:30,  9.86it/s, est. speed input: 3615.55 toks/s, output: 880.70 toks/s]
Processed prompts:  85%|████████▌ | 8515/10000 [12:20<02:27, 10.07it/s, est. speed input: 3615.23 toks/s, output: 880.70 toks/s]
Processed prompts:  85%|████████▌ | 8517/10000 [12:20<03:22,  7.32it/s, est. speed input: 3613.30 toks/s, output: 880.33 toks/s]
Processed prompts:  85%|████████▌ | 8525/10000 [12:20<01:46, 13.91it/s, est. speed input: 3614.57 toks/s, output: 880.86 toks/s]
Processed prompts:  85%|████████▌ | 8528/10000 [12:21<02:31,  9.74it/s, est. speed input: 3612.46 toks/s, output: 880.45 toks/s]
Processed prompts:  85%|████████▌ | 8530/10000 [12:21<02:46,  8.83it/s, est. speed input: 3611.48 toks/s, output: 880.27 toks/s]
Processed prompts:  85%|████████▌ | 8532/10000 [12:22<02:48,  8.73it/s, est. speed input: 3610.90 toks/s, output: 880.18 toks/s]
Processed prompts:  85%|████████▌ | 8541/10000 [12:22<01:34, 15.43it/s, est. speed input: 3612.26 toks/s, output: 880.72 toks/s]
Processed prompts:  85%|████████▌ | 8544/10000 [12:22<02:19, 10.44it/s, est. speed input: 3610.09 toks/s, output: 880.30 toks/s]
Processed prompts:  85%|████████▌ | 8546/10000 [12:23<02:30,  9.64it/s, est. speed input: 3609.27 toks/s, output: 880.17 toks/s]
Processed prompts:  85%|████████▌ | 8548/10000 [12:23<02:39,  9.13it/s, est. speed input: 3608.53 toks/s, output: 880.05 toks/s]
Processed prompts:  86%|████████▌ | 8553/10000 [12:23<02:01, 11.87it/s, est. speed input: 3608.77 toks/s, output: 880.25 toks/s]
Processed prompts:  86%|████████▌ | 8559/10000 [12:24<01:51, 12.96it/s, est. speed input: 3608.54 toks/s, output: 880.40 toks/s]
Processed prompts:  86%|████████▌ | 8564/10000 [12:24<01:58, 12.13it/s, est. speed input: 3607.74 toks/s, output: 880.30 toks/s]
Processed prompts:  86%|████████▌ | 8567/10000 [12:24<02:12, 10.80it/s, est. speed input: 3606.71 toks/s, output: 880.14 toks/s]
Processed prompts:  86%|████████▌ | 8570/10000 [12:25<02:56,  8.12it/s, est. speed input: 3604.34 toks/s, output: 879.63 toks/s]
Processed prompts:  86%|████████▌ | 8572/10000 [12:25<02:41,  8.86it/s, est. speed input: 3604.23 toks/s, output: 879.66 toks/s]
Processed prompts:  86%|████████▌ | 8575/10000 [12:26<02:28,  9.60it/s, est. speed input: 3603.89 toks/s, output: 879.65 toks/s]
Processed prompts:  86%|████████▌ | 8579/10000 [12:26<02:12, 10.73it/s, est. speed input: 3603.62 toks/s, output: 879.69 toks/s]
Processed prompts:  86%|████████▌ | 8582/10000 [12:26<02:22,  9.96it/s, est. speed input: 3602.77 toks/s, output: 879.56 toks/s]
Processed prompts:  86%|████████▌ | 8585/10000 [12:26<02:20, 10.09it/s, est. speed input: 3602.25 toks/s, output: 879.55 toks/s]
Processed prompts:  86%|████████▌ | 8587/10000 [12:27<02:38,  8.90it/s, est. speed input: 3601.25 toks/s, output: 879.38 toks/s]
Processed prompts:  86%|████████▌ | 8590/10000 [12:27<02:24,  9.72it/s, est. speed input: 3600.90 toks/s, output: 879.36 toks/s]
Processed prompts:  86%|████████▌ | 8595/10000 [12:27<01:57, 11.94it/s, est. speed input: 3600.90 toks/s, output: 879.47 toks/s]
Processed prompts:  86%|████████▌ | 8603/10000 [12:28<01:32, 15.10it/s, est. speed input: 3601.29 toks/s, output: 879.77 toks/s]
Processed prompts:  86%|████████▌ | 8609/10000 [12:28<01:45, 13.18it/s, est. speed input: 3600.28 toks/s, output: 879.62 toks/s]
Processed prompts:  86%|████████▌ | 8612/10000 [12:29<02:04, 11.14it/s, est. speed input: 3599.00 toks/s, output: 879.40 toks/s]
Processed prompts:  86%|████████▌ | 8618/10000 [12:29<01:44, 13.17it/s, est. speed input: 3599.19 toks/s, output: 879.56 toks/s]
Processed prompts:  86%|████████▌ | 8622/10000 [12:30<02:03, 11.20it/s, est. speed input: 3597.84 toks/s, output: 879.34 toks/s]
Processed prompts:  86%|████████▌ | 8624/10000 [12:30<02:24,  9.54it/s, est. speed input: 3596.59 toks/s, output: 879.09 toks/s]
Processed prompts:  86%|████████▋ | 8626/10000 [12:30<02:29,  9.22it/s, est. speed input: 3595.96 toks/s, output: 878.99 toks/s]
Processed prompts:  86%|████████▋ | 8628/10000 [12:30<02:35,  8.81it/s, est. speed input: 3595.24 toks/s, output: 878.88 toks/s]
Processed prompts:  86%|████████▋ | 8632/10000 [12:31<02:08, 10.63it/s, est. speed input: 3595.17 toks/s, output: 878.99 toks/s]
Processed prompts:  86%|████████▋ | 8637/10000 [12:31<01:56, 11.65it/s, est. speed input: 3594.80 toks/s, output: 879.01 toks/s]
Processed prompts:  86%|████████▋ | 8640/10000 [12:31<02:17,  9.88it/s, est. speed input: 3593.55 toks/s, output: 878.79 toks/s]
Processed prompts:  86%|████████▋ | 8642/10000 [12:32<02:29,  9.11it/s, est. speed input: 3592.78 toks/s, output: 878.66 toks/s]
Processed prompts:  86%|████████▋ | 8643/10000 [12:32<02:43,  8.30it/s, est. speed input: 3592.12 toks/s, output: 878.53 toks/s]
Processed prompts:  86%|████████▋ | 8645/10000 [12:32<02:44,  8.25it/s, est. speed input: 3591.50 toks/s, output: 878.43 toks/s]
Processed prompts:  87%|████████▋ | 8651/10000 [12:32<01:46, 12.68it/s, est. speed input: 3592.02 toks/s, output: 878.73 toks/s]
Processed prompts:  87%|████████▋ | 8658/10000 [12:33<01:39, 13.53it/s, est. speed input: 3591.70 toks/s, output: 878.86 toks/s]
Processed prompts:  87%|████████▋ | 8662/10000 [12:33<02:00, 11.13it/s, est. speed input: 3590.30 toks/s, output: 878.63 toks/s]
Processed prompts:  87%|████████▋ | 8664/10000 [12:34<02:24,  9.23it/s, est. speed input: 3588.98 toks/s, output: 878.36 toks/s]
Processed prompts:  87%|████████▋ | 8669/10000 [12:34<01:57, 11.35it/s, est. speed input: 3589.09 toks/s, output: 878.51 toks/s]
Processed prompts:  87%|████████▋ | 8674/10000 [12:35<01:56, 11.37it/s, est. speed input: 3588.38 toks/s, output: 878.49 toks/s]
Processed prompts:  87%|████████▋ | 8678/10000 [12:35<02:02, 10.82it/s, est. speed input: 3587.53 toks/s, output: 878.40 toks/s]
Processed prompts:  87%|████████▋ | 8683/10000 [12:35<01:52, 11.69it/s, est. speed input: 3587.21 toks/s, output: 878.46 toks/s]
Processed prompts:  87%|████████▋ | 8685/10000 [12:36<02:17,  9.55it/s, est. speed input: 3585.81 toks/s, output: 878.17 toks/s]
Processed prompts:  87%|████████▋ | 8688/10000 [12:36<02:09, 10.14it/s, est. speed input: 3585.51 toks/s, output: 878.15 toks/s]
Processed prompts:  87%|████████▋ | 8692/10000 [12:36<02:01, 10.79it/s, est. speed input: 3585.13 toks/s, output: 878.16 toks/s]
Processed prompts:  87%|████████▋ | 8695/10000 [12:37<02:11,  9.95it/s, est. speed input: 3584.26 toks/s, output: 878.04 toks/s]
Processed prompts:  87%|████████▋ | 8701/10000 [12:37<01:39, 13.03it/s, est. speed input: 3584.68 toks/s, output: 878.32 toks/s]
Processed prompts:  87%|████████▋ | 8703/10000 [12:37<02:17,  9.46it/s, est. speed input: 3582.87 toks/s, output: 877.94 toks/s]
Processed prompts:  87%|████████▋ | 8706/10000 [12:38<02:09,  9.98it/s, est. speed input: 3582.50 toks/s, output: 877.94 toks/s]
Processed prompts:  87%|████████▋ | 8708/10000 [12:38<02:20,  9.16it/s, est. speed input: 3581.67 toks/s, output: 877.79 toks/s]
Processed prompts:  87%|████████▋ | 8710/10000 [12:38<02:25,  8.84it/s, est. speed input: 3581.10 toks/s, output: 877.71 toks/s]
Processed prompts:  87%|████████▋ | 8715/10000 [12:38<01:33, 13.78it/s, est. speed input: 3582.40 toks/s, output: 878.02 toks/s]
Processed prompts:  87%|████████▋ | 8718/10000 [12:39<01:29, 14.27it/s, est. speed input: 3582.33 toks/s, output: 878.09 toks/s]
Processed prompts:  87%|████████▋ | 8724/10000 [12:39<01:13, 17.44it/s, est. speed input: 3582.84 toks/s, output: 878.37 toks/s]
Processed prompts:  87%|████████▋ | 8727/10000 [12:39<01:11, 17.84it/s, est. speed input: 3582.96 toks/s, output: 878.48 toks/s]
Processed prompts:  87%|████████▋ | 8734/10000 [12:39<00:59, 21.28it/s, est. speed input: 3583.70 toks/s, output: 878.87 toks/s]
Processed prompts:  87%|████████▋ | 8739/10000 [12:39<00:53, 23.51it/s, est. speed input: 3584.60 toks/s, output: 879.16 toks/s]
Processed prompts:  87%|████████▋ | 8742/10000 [12:40<01:14, 16.92it/s, est. speed input: 3583.72 toks/s, output: 879.06 toks/s]
Processed prompts:  87%|████████▋ | 8747/10000 [12:40<01:15, 16.53it/s, est. speed input: 3583.96 toks/s, output: 879.18 toks/s]
Processed prompts:  88%|████████▊ | 8752/10000 [12:40<01:00, 20.54it/s, est. speed input: 3585.30 toks/s, output: 879.51 toks/s]
Processed prompts:  88%|████████▊ | 8755/10000 [12:40<00:59, 21.07it/s, est. speed input: 3585.61 toks/s, output: 879.68 toks/s]
Processed prompts:  88%|████████▊ | 8758/10000 [12:40<00:54, 22.62it/s, est. speed input: 3586.64 toks/s, output: 879.86 toks/s]
Processed prompts:  88%|████████▊ | 8761/10000 [12:41<01:05, 18.88it/s, est. speed input: 3586.67 toks/s, output: 879.88 toks/s]
Processed prompts:  88%|████████▊ | 8766/10000 [12:41<01:00, 20.26it/s, est. speed input: 3587.80 toks/s, output: 880.11 toks/s]
Processed prompts:  88%|████████▊ | 8771/10000 [12:41<00:48, 25.17it/s, est. speed input: 3590.15 toks/s, output: 880.45 toks/s]
Processed prompts:  88%|████████▊ | 8774/10000 [12:41<01:10, 17.31it/s, est. speed input: 3590.36 toks/s, output: 880.33 toks/s]
Processed prompts:  88%|████████▊ | 8782/10000 [12:42<00:54, 22.51it/s, est. speed input: 3593.54 toks/s, output: 880.85 toks/s]
Processed prompts:  88%|████████▊ | 8785/10000 [12:42<00:55, 21.77it/s, est. speed input: 3594.76 toks/s, output: 880.97 toks/s]
Processed prompts:  88%|████████▊ | 8789/10000 [12:42<00:51, 23.63it/s, est. speed input: 3596.26 toks/s, output: 881.23 toks/s]
Processed prompts:  88%|████████▊ | 8792/10000 [12:42<01:02, 19.23it/s, est. speed input: 3596.84 toks/s, output: 881.25 toks/s]
Processed prompts:  88%|████████▊ | 8796/10000 [12:42<01:06, 18.22it/s, est. speed input: 3597.84 toks/s, output: 881.36 toks/s]
Processed prompts:  88%|████████▊ | 8799/10000 [12:43<01:13, 16.32it/s, est. speed input: 3598.11 toks/s, output: 881.37 toks/s]
Processed prompts:  88%|████████▊ | 8806/10000 [12:43<00:57, 20.76it/s, est. speed input: 3600.97 toks/s, output: 881.84 toks/s]
Processed prompts:  88%|████████▊ | 8810/10000 [12:43<00:52, 22.73it/s, est. speed input: 3602.48 toks/s, output: 882.11 toks/s]
Processed prompts:  88%|████████▊ | 8814/10000 [12:43<00:51, 23.08it/s, est. speed input: 3603.95 toks/s, output: 882.34 toks/s]
Processed prompts:  88%|████████▊ | 8819/10000 [12:43<00:46, 25.35it/s, est. speed input: 3605.72 toks/s, output: 882.67 toks/s]
Processed prompts:  88%|████████▊ | 8822/10000 [12:43<00:51, 22.79it/s, est. speed input: 3606.56 toks/s, output: 882.76 toks/s]
Processed prompts:  88%|████████▊ | 8825/10000 [12:44<00:51, 22.82it/s, est. speed input: 3607.29 toks/s, output: 882.92 toks/s]
Processed prompts:  88%|████████▊ | 8832/10000 [12:44<00:36, 32.28it/s, est. speed input: 3610.09 toks/s, output: 883.51 toks/s]
Processed prompts:  88%|████████▊ | 8836/10000 [12:44<00:47, 24.73it/s, est. speed input: 3611.15 toks/s, output: 883.63 toks/s]
Processed prompts:  88%|████████▊ | 8840/10000 [12:44<00:46, 24.93it/s, est. speed input: 3612.44 toks/s, output: 883.86 toks/s]
Processed prompts:  88%|████████▊ | 8843/10000 [12:44<01:00, 19.22it/s, est. speed input: 3612.39 toks/s, output: 883.87 toks/s]
Processed prompts:  88%|████████▊ | 8850/10000 [12:45<00:48, 23.66it/s, est. speed input: 3615.08 toks/s, output: 884.32 toks/s]
Processed prompts:  89%|████████▊ | 8853/10000 [12:45<00:50, 22.65it/s, est. speed input: 3615.81 toks/s, output: 884.49 toks/s]
Processed prompts:  89%|████████▊ | 8858/10000 [12:45<00:41, 27.40it/s, est. speed input: 3617.76 toks/s, output: 884.87 toks/s]
Processed prompts:  89%|████████▊ | 8862/10000 [12:45<00:46, 24.49it/s, est. speed input: 3618.65 toks/s, output: 885.04 toks/s]
Processed prompts:  89%|████████▊ | 8865/10000 [12:45<00:49, 22.97it/s, est. speed input: 3619.56 toks/s, output: 885.17 toks/s]
Processed prompts:  89%|████████▊ | 8869/10000 [12:45<00:44, 25.19it/s, est. speed input: 3621.11 toks/s, output: 885.44 toks/s]
Processed prompts:  89%|████████▊ | 8874/10000 [12:45<00:42, 26.78it/s, est. speed input: 3623.06 toks/s, output: 885.80 toks/s]
Processed prompts:  89%|████████▉ | 8877/10000 [12:46<00:47, 23.66it/s, est. speed input: 3623.98 toks/s, output: 885.88 toks/s]
Processed prompts:  89%|████████▉ | 8880/10000 [12:46<00:48, 23.31it/s, est. speed input: 3625.03 toks/s, output: 885.99 toks/s]
Processed prompts:  89%|████████▉ | 8883/10000 [12:46<00:48, 23.17it/s, est. speed input: 3625.97 toks/s, output: 886.20 toks/s]
Processed prompts:  89%|████████▉ | 8891/10000 [12:46<00:33, 33.19it/s, est. speed input: 3629.17 toks/s, output: 886.89 toks/s]
Processed prompts:  89%|████████▉ | 8896/10000 [12:46<00:41, 26.47it/s, est. speed input: 3630.19 toks/s, output: 887.09 toks/s]
Processed prompts:  89%|████████▉ | 8899/10000 [12:47<00:46, 23.74it/s, est. speed input: 3631.39 toks/s, output: 887.18 toks/s]
Processed prompts:  89%|████████▉ | 8904/10000 [12:47<00:40, 27.01it/s, est. speed input: 3633.05 toks/s, output: 887.55 toks/s]
Processed prompts:  89%|████████▉ | 8907/10000 [12:47<00:45, 23.87it/s, est. speed input: 3633.97 toks/s, output: 887.67 toks/s]
Processed prompts:  89%|████████▉ | 8917/10000 [12:47<00:50, 21.60it/s, est. speed input: 3636.77 toks/s, output: 888.08 toks/s]
Processed prompts:  89%|████████▉ | 8921/10000 [12:47<00:46, 22.98it/s, est. speed input: 3638.03 toks/s, output: 888.37 toks/s]
Processed prompts:  89%|████████▉ | 8924/10000 [12:48<00:44, 24.03it/s, est. speed input: 3639.05 toks/s, output: 888.61 toks/s]
Processed prompts:  89%|████████▉ | 8927/10000 [12:48<00:45, 23.76it/s, est. speed input: 3640.17 toks/s, output: 888.81 toks/s]
Processed prompts:  89%|████████▉ | 8933/10000 [12:48<00:35, 30.21it/s, est. speed input: 3643.41 toks/s, output: 889.23 toks/s]
Processed prompts:  89%|████████▉ | 8937/10000 [12:48<00:52, 20.15it/s, est. speed input: 3643.92 toks/s, output: 889.22 toks/s]
Processed prompts:  89%|████████▉ | 8944/10000 [12:48<00:41, 25.38it/s, est. speed input: 3646.69 toks/s, output: 889.77 toks/s]
Processed prompts:  90%|████████▉ | 8952/10000 [12:49<00:34, 30.27it/s, est. speed input: 3649.85 toks/s, output: 890.43 toks/s]
Processed prompts:  90%|████████▉ | 8956/10000 [12:49<00:42, 24.48it/s, est. speed input: 3650.70 toks/s, output: 890.51 toks/s]
Processed prompts:  90%|████████▉ | 8962/10000 [12:49<00:38, 27.05it/s, est. speed input: 3652.98 toks/s, output: 890.86 toks/s]
Processed prompts:  90%|████████▉ | 8966/10000 [12:49<00:46, 22.14it/s, est. speed input: 3653.56 toks/s, output: 890.94 toks/s]
Processed prompts:  90%|████████▉ | 8969/10000 [12:49<00:46, 22.12it/s, est. speed input: 3654.63 toks/s, output: 891.09 toks/s]
Processed prompts:  90%|████████▉ | 8973/10000 [12:50<00:41, 24.90it/s, est. speed input: 3656.31 toks/s, output: 891.35 toks/s]
Processed prompts:  90%|████████▉ | 8976/10000 [12:50<00:45, 22.44it/s, est. speed input: 3657.05 toks/s, output: 891.47 toks/s]
Processed prompts:  90%|████████▉ | 8979/10000 [12:50<00:56, 18.19it/s, est. speed input: 3657.06 toks/s, output: 891.54 toks/s]
Processed prompts:  90%|████████▉ | 8987/10000 [12:50<00:44, 22.84it/s, est. speed input: 3660.15 toks/s, output: 892.13 toks/s]
Processed prompts:  90%|████████▉ | 8990/10000 [12:50<00:42, 23.79it/s, est. speed input: 3661.48 toks/s, output: 892.37 toks/s]
Processed prompts:  90%|████████▉ | 8993/10000 [12:51<00:53, 18.78it/s, est. speed input: 3661.69 toks/s, output: 892.38 toks/s]
Processed prompts:  90%|████████▉ | 8999/10000 [12:51<00:46, 21.58it/s, est. speed input: 3663.70 toks/s, output: 892.72 toks/s]
Processed prompts:  90%|█████████ | 9002/10000 [12:51<00:45, 22.08it/s, est. speed input: 3664.43 toks/s, output: 892.90 toks/s]
Processed prompts:  90%|█████████ | 9005/10000 [12:51<00:53, 18.75it/s, est. speed input: 3664.92 toks/s, output: 892.91 toks/s]
Processed prompts:  90%|█████████ | 9009/10000 [12:51<00:50, 19.63it/s, est. speed input: 3666.05 toks/s, output: 893.19 toks/s]
Processed prompts:  90%|█████████ | 9013/10000 [12:51<00:42, 23.03it/s, est. speed input: 3667.52 toks/s, output: 893.52 toks/s]
Processed prompts:  90%|█████████ | 9016/10000 [12:52<00:44, 21.89it/s, est. speed input: 3668.30 toks/s, output: 893.64 toks/s]
Processed prompts:  90%|█████████ | 9019/10000 [12:52<00:57, 17.12it/s, est. speed input: 3668.22 toks/s, output: 893.60 toks/s]
Processed prompts:  90%|█████████ | 9023/10000 [12:52<00:51, 18.90it/s, est. speed input: 3669.35 toks/s, output: 893.80 toks/s]
Processed prompts:  90%|█████████ | 9029/10000 [12:52<00:50, 19.33it/s, est. speed input: 3670.71 toks/s, output: 894.00 toks/s]
Processed prompts:  90%|█████████ | 9037/10000 [12:53<00:40, 24.06it/s, est. speed input: 3674.02 toks/s, output: 894.49 toks/s]
Processed prompts:  90%|█████████ | 9040/10000 [12:53<00:45, 21.16it/s, est. speed input: 3674.57 toks/s, output: 894.56 toks/s]
Processed prompts:  90%|█████████ | 9043/10000 [12:53<00:43, 22.11it/s, est. speed input: 3675.49 toks/s, output: 894.83 toks/s]
Processed prompts:  90%|█████████ | 9046/10000 [12:53<00:40, 23.30it/s, est. speed input: 3676.73 toks/s, output: 895.01 toks/s]
Processed prompts:  90%|█████████ | 9049/10000 [12:53<00:43, 22.01it/s, est. speed input: 3677.24 toks/s, output: 895.12 toks/s]
Processed prompts:  91%|█████████ | 9052/10000 [12:53<00:49, 19.17it/s, est. speed input: 3677.73 toks/s, output: 895.23 toks/s]
Processed prompts:  91%|█████████ | 9056/10000 [12:54<00:51, 18.37it/s, est. speed input: 3678.71 toks/s, output: 895.39 toks/s]
Processed prompts:  91%|█████████ | 9064/10000 [12:54<00:44, 21.09it/s, est. speed input: 3681.34 toks/s, output: 895.86 toks/s]
Processed prompts:  91%|█████████ | 9070/10000 [12:54<00:36, 25.14it/s, est. speed input: 3683.59 toks/s, output: 896.32 toks/s]
Processed prompts:  91%|█████████ | 9075/10000 [12:54<00:36, 25.36it/s, est. speed input: 3685.29 toks/s, output: 896.72 toks/s]
Processed prompts:  91%|█████████ | 9078/10000 [12:55<00:45, 20.47it/s, est. speed input: 3685.50 toks/s, output: 896.82 toks/s]
Processed prompts:  91%|█████████ | 9082/10000 [12:55<00:38, 23.58it/s, est. speed input: 3687.19 toks/s, output: 897.16 toks/s]
Processed prompts:  91%|█████████ | 9087/10000 [12:55<00:35, 25.76it/s, est. speed input: 3688.95 toks/s, output: 897.51 toks/s]
Processed prompts:  91%|█████████ | 9091/10000 [12:55<00:36, 24.63it/s, est. speed input: 3690.43 toks/s, output: 897.70 toks/s]
Processed prompts:  91%|█████████ | 9094/10000 [12:55<00:45, 19.96it/s, est. speed input: 3690.89 toks/s, output: 897.72 toks/s]
Processed prompts:  91%|█████████ | 9097/10000 [12:55<00:41, 21.54it/s, est. speed input: 3692.12 toks/s, output: 897.95 toks/s]
Processed prompts:  91%|█████████ | 9105/10000 [12:55<00:28, 31.37it/s, est. speed input: 3695.52 toks/s, output: 898.61 toks/s]
Processed prompts:  91%|█████████ | 9109/10000 [12:56<00:44, 20.14it/s, est. speed input: 3695.64 toks/s, output: 898.56 toks/s]
Processed prompts:  91%|█████████ | 9117/10000 [12:56<00:34, 25.55it/s, est. speed input: 3698.81 toks/s, output: 899.11 toks/s]
Processed prompts:  91%|█████████ | 9121/10000 [12:56<00:42, 20.58it/s, est. speed input: 3699.44 toks/s, output: 899.17 toks/s]
Processed prompts:  91%|█████████▏| 9128/10000 [12:57<00:38, 22.69it/s, est. speed input: 3701.76 toks/s, output: 899.64 toks/s]
Processed prompts:  91%|█████████▏| 9133/10000 [12:57<00:32, 26.39it/s, est. speed input: 3704.01 toks/s, output: 899.98 toks/s]
Processed prompts:  91%|█████████▏| 9140/10000 [12:57<00:28, 29.89it/s, est. speed input: 3706.57 toks/s, output: 900.53 toks/s]
Processed prompts:  91%|█████████▏| 9144/10000 [12:57<00:33, 25.19it/s, est. speed input: 3707.69 toks/s, output: 900.67 toks/s]
Processed prompts:  91%|█████████▏| 9149/10000 [12:57<00:32, 26.47it/s, est. speed input: 3709.61 toks/s, output: 901.03 toks/s]
Processed prompts:  92%|█████████▏| 9152/10000 [12:58<00:35, 23.79it/s, est. speed input: 3710.31 toks/s, output: 901.12 toks/s]
Processed prompts:  92%|█████████▏| 9155/10000 [12:58<00:34, 24.63it/s, est. speed input: 3711.41 toks/s, output: 901.26 toks/s]
Processed prompts:  92%|█████████▏| 9158/10000 [12:58<00:34, 24.13it/s, est. speed input: 3712.46 toks/s, output: 901.47 toks/s]
Processed prompts:  92%|█████████▏| 9161/10000 [12:58<00:33, 24.68it/s, est. speed input: 3713.54 toks/s, output: 901.64 toks/s]
Processed prompts:  92%|█████████▏| 9164/10000 [12:58<00:38, 21.92it/s, est. speed input: 3714.30 toks/s, output: 901.77 toks/s]
Processed prompts:  92%|█████████▏| 9167/10000 [12:58<00:45, 18.16it/s, est. speed input: 3714.90 toks/s, output: 901.83 toks/s]
Processed prompts:  92%|█████████▏| 9173/10000 [12:58<00:37, 22.30it/s, est. speed input: 3716.95 toks/s, output: 902.22 toks/s]
Processed prompts:  92%|█████████▏| 9176/10000 [12:59<00:43, 18.97it/s, est. speed input: 3717.57 toks/s, output: 902.30 toks/s]
Processed prompts:  92%|█████████▏| 9186/10000 [12:59<00:33, 24.47it/s, est. speed input: 3721.55 toks/s, output: 903.08 toks/s]
Processed prompts:  92%|█████████▏| 9189/10000 [12:59<00:35, 23.02it/s, est. speed input: 3722.50 toks/s, output: 903.23 toks/s]
Processed prompts:  92%|█████████▏| 9195/10000 [12:59<00:29, 27.57it/s, est. speed input: 3725.19 toks/s, output: 903.78 toks/s]
Processed prompts:  92%|█████████▏| 9198/10000 [13:00<00:35, 22.57it/s, est. speed input: 3725.75 toks/s, output: 903.82 toks/s]
Processed prompts:  92%|█████████▏| 9201/10000 [13:00<00:35, 22.65it/s, est. speed input: 3726.69 toks/s, output: 904.03 toks/s]
Processed prompts:  92%|█████████▏| 9207/10000 [13:00<00:26, 29.49it/s, est. speed input: 3729.47 toks/s, output: 904.53 toks/s]
Processed prompts:  92%|█████████▏| 9211/10000 [13:00<00:32, 24.21it/s, est. speed input: 3730.32 toks/s, output: 904.68 toks/s]
Processed prompts:  92%|█████████▏| 9214/10000 [13:00<00:41, 19.14it/s, est. speed input: 3730.71 toks/s, output: 904.70 toks/s]
Processed prompts:  92%|█████████▏| 9218/10000 [13:01<00:41, 19.04it/s, est. speed input: 3731.73 toks/s, output: 904.87 toks/s]
Processed prompts:  92%|█████████▏| 9225/10000 [13:01<00:32, 24.09it/s, est. speed input: 3734.56 toks/s, output: 905.52 toks/s]
Processed prompts:  92%|█████████▏| 9231/10000 [13:01<00:29, 26.39it/s, est. speed input: 3736.63 toks/s, output: 905.96 toks/s]
Processed prompts:  92%|█████████▏| 9235/10000 [13:01<00:32, 23.82it/s, est. speed input: 3737.72 toks/s, output: 906.13 toks/s]
Processed prompts:  92%|█████████▏| 9238/10000 [13:01<00:33, 22.62it/s, est. speed input: 3738.38 toks/s, output: 906.24 toks/s]
Processed prompts:  92%|█████████▏| 9241/10000 [13:01<00:38, 19.67it/s, est. speed input: 3739.07 toks/s, output: 906.29 toks/s]
Processed prompts:  92%|█████████▏| 9245/10000 [13:02<00:38, 19.42it/s, est. speed input: 3740.33 toks/s, output: 906.46 toks/s]
Processed prompts:  92%|█████████▏| 9248/10000 [13:02<00:35, 21.07it/s, est. speed input: 3741.36 toks/s, output: 906.69 toks/s]
Processed prompts:  93%|█████████▎| 9251/10000 [13:02<00:33, 22.53it/s, est. speed input: 3742.58 toks/s, output: 906.90 toks/s]
Processed prompts:  93%|█████████▎| 9255/10000 [13:02<00:30, 24.67it/s, est. speed input: 3743.90 toks/s, output: 907.18 toks/s]
Processed prompts:  93%|█████████▎| 9258/10000 [13:02<00:33, 21.91it/s, est. speed input: 3744.53 toks/s, output: 907.30 toks/s]
Processed prompts:  93%|█████████▎| 9261/10000 [13:02<00:33, 21.94it/s, est. speed input: 3745.43 toks/s, output: 907.48 toks/s]
Processed prompts:  93%|█████████▎| 9264/10000 [13:02<00:33, 21.92it/s, est. speed input: 3746.50 toks/s, output: 907.65 toks/s]
Processed prompts:  93%|█████████▎| 9267/10000 [13:03<00:38, 19.16it/s, est. speed input: 3747.05 toks/s, output: 907.78 toks/s]
Processed prompts:  93%|█████████▎| 9272/10000 [13:03<00:29, 24.81it/s, est. speed input: 3749.13 toks/s, output: 908.14 toks/s]
Processed prompts:  93%|█████████▎| 9275/10000 [13:03<00:32, 22.01it/s, est. speed input: 3749.86 toks/s, output: 908.18 toks/s]
Processed prompts:  93%|█████████▎| 9279/10000 [13:03<00:34, 21.12it/s, est. speed input: 3751.01 toks/s, output: 908.35 toks/s]
Processed prompts:  93%|█████████▎| 9282/10000 [13:03<00:33, 21.34it/s, est. speed input: 3751.98 toks/s, output: 908.53 toks/s]
Processed prompts:  93%|█████████▎| 9285/10000 [13:04<00:40, 17.83it/s, est. speed input: 3752.31 toks/s, output: 908.56 toks/s]
Processed prompts:  93%|█████████▎| 9293/10000 [13:04<00:30, 23.33it/s, est. speed input: 3755.03 toks/s, output: 909.09 toks/s]
Processed prompts:  93%|█████████▎| 9296/10000 [13:04<00:39, 17.81it/s, est. speed input: 3754.94 toks/s, output: 909.05 toks/s]
Processed prompts:  93%|█████████▎| 9300/10000 [13:04<00:36, 19.04it/s, est. speed input: 3756.20 toks/s, output: 909.29 toks/s]
Processed prompts:  93%|█████████▎| 9304/10000 [13:04<00:31, 21.92it/s, est. speed input: 3757.73 toks/s, output: 909.60 toks/s]
Processed prompts:  93%|█████████▎| 9308/10000 [13:05<00:29, 23.07it/s, est. speed input: 3759.27 toks/s, output: 909.85 toks/s]
Processed prompts:  93%|█████████▎| 9312/10000 [13:05<00:28, 23.86it/s, est. speed input: 3760.74 toks/s, output: 910.09 toks/s]
Processed prompts:  93%|█████████▎| 9315/10000 [13:05<00:30, 22.47it/s, est. speed input: 3761.71 toks/s, output: 910.26 toks/s]
Processed prompts:  93%|█████████▎| 9318/10000 [13:05<00:29, 23.36it/s, est. speed input: 3762.71 toks/s, output: 910.48 toks/s]
Processed prompts:  93%|█████████▎| 9321/10000 [13:05<00:29, 23.19it/s, est. speed input: 3763.74 toks/s, output: 910.68 toks/s]
Processed prompts:  93%|█████████▎| 9325/10000 [13:05<00:26, 25.23it/s, est. speed input: 3765.06 toks/s, output: 910.96 toks/s]
Processed prompts:  93%|█████████▎| 9329/10000 [13:05<00:28, 23.90it/s, est. speed input: 3766.11 toks/s, output: 911.13 toks/s]
Processed prompts:  93%|█████████▎| 9333/10000 [13:06<00:27, 24.24it/s, est. speed input: 3767.53 toks/s, output: 911.35 toks/s]
Processed prompts:  93%|█████████▎| 9337/10000 [13:06<00:27, 24.30it/s, est. speed input: 3768.94 toks/s, output: 911.56 toks/s]
Processed prompts:  93%|█████████▎| 9340/10000 [13:06<00:29, 22.71it/s, est. speed input: 3769.90 toks/s, output: 911.67 toks/s]
Processed prompts:  93%|█████████▎| 9347/10000 [13:06<00:26, 24.41it/s, est. speed input: 3772.34 toks/s, output: 912.03 toks/s]
Processed prompts:  94%|█████████▎| 9350/10000 [13:06<00:28, 22.74it/s, est. speed input: 3772.92 toks/s, output: 912.15 toks/s]
Processed prompts:  94%|█████████▎| 9353/10000 [13:07<00:33, 19.38it/s, est. speed input: 3773.37 toks/s, output: 912.18 toks/s]
Processed prompts:  94%|█████████▎| 9363/10000 [13:07<00:24, 26.44it/s, est. speed input: 3777.51 toks/s, output: 912.94 toks/s]
Processed prompts:  94%|█████████▎| 9366/10000 [13:07<00:32, 19.73it/s, est. speed input: 3777.38 toks/s, output: 912.91 toks/s]
Processed prompts:  94%|█████████▎| 9369/10000 [13:07<00:32, 19.55it/s, est. speed input: 3778.15 toks/s, output: 913.05 toks/s]
Processed prompts:  94%|█████████▎| 9372/10000 [13:07<00:29, 21.06it/s, est. speed input: 3779.00 toks/s, output: 913.25 toks/s]
Processed prompts:  94%|█████████▍| 9376/10000 [13:08<00:27, 23.01it/s, est. speed input: 3780.34 toks/s, output: 913.47 toks/s]
Processed prompts:  94%|█████████▍| 9379/10000 [13:08<00:28, 21.58it/s, est. speed input: 3781.03 toks/s, output: 913.59 toks/s]
Processed prompts:  94%|█████████▍| 9383/10000 [13:08<00:30, 19.97it/s, est. speed input: 3782.04 toks/s, output: 913.77 toks/s]
Processed prompts:  94%|█████████▍| 9388/10000 [13:08<00:24, 25.30it/s, est. speed input: 3784.04 toks/s, output: 914.21 toks/s]
Processed prompts:  94%|█████████▍| 9394/10000 [13:08<00:21, 27.93it/s, est. speed input: 3786.61 toks/s, output: 914.64 toks/s]
Processed prompts:  94%|█████████▍| 9398/10000 [13:08<00:24, 24.87it/s, est. speed input: 3787.68 toks/s, output: 914.81 toks/s]
Processed prompts:  94%|█████████▍| 9401/10000 [13:09<00:24, 24.09it/s, est. speed input: 3788.55 toks/s, output: 914.95 toks/s]
Processed prompts:  94%|█████████▍| 9404/10000 [13:09<00:30, 19.76it/s, est. speed input: 3789.12 toks/s, output: 915.01 toks/s]
Processed prompts:  94%|█████████▍| 9409/10000 [13:09<00:23, 24.77it/s, est. speed input: 3791.24 toks/s, output: 915.40 toks/s]
Processed prompts:  94%|█████████▍| 9412/10000 [13:09<00:26, 22.22it/s, est. speed input: 3792.14 toks/s, output: 915.54 toks/s]
Processed prompts:  94%|█████████▍| 9415/10000 [13:09<00:26, 22.40it/s, est. speed input: 3793.11 toks/s, output: 915.73 toks/s]
Processed prompts:  94%|█████████▍| 9418/10000 [13:09<00:24, 23.35it/s, est. speed input: 3793.91 toks/s, output: 915.93 toks/s]
Processed prompts:  94%|█████████▍| 9421/10000 [13:10<00:26, 22.16it/s, est. speed input: 3794.83 toks/s, output: 916.12 toks/s]
Processed prompts:  94%|█████████▍| 9425/10000 [13:10<00:22, 25.79it/s, est. speed input: 3796.33 toks/s, output: 916.43 toks/s]
Processed prompts:  94%|█████████▍| 9429/10000 [13:10<00:30, 18.45it/s, est. speed input: 3796.68 toks/s, output: 916.47 toks/s]
Processed prompts:  94%|█████████▍| 9435/10000 [13:10<00:25, 22.06it/s, est. speed input: 3799.10 toks/s, output: 916.94 toks/s]
Processed prompts:  94%|█████████▍| 9438/10000 [13:10<00:24, 22.93it/s, est. speed input: 3800.20 toks/s, output: 917.12 toks/s]
Processed prompts:  94%|█████████▍| 9441/10000 [13:10<00:23, 24.07it/s, est. speed input: 3801.40 toks/s, output: 917.32 toks/s]
Processed prompts:  94%|█████████▍| 9444/10000 [13:11<00:22, 24.65it/s, est. speed input: 3802.51 toks/s, output: 917.50 toks/s]
Processed prompts:  94%|█████████▍| 9447/10000 [13:11<00:22, 24.53it/s, est. speed input: 3803.62 toks/s, output: 917.69 toks/s]
Processed prompts:  94%|█████████▍| 9450/10000 [13:11<00:22, 24.06it/s, est. speed input: 3804.45 toks/s, output: 917.85 toks/s]
Processed prompts:  95%|█████████▍| 9453/10000 [13:11<00:23, 23.71it/s, est. speed input: 3805.41 toks/s, output: 918.00 toks/s]
Processed prompts:  95%|█████████▍| 9456/10000 [13:11<00:24, 22.16it/s, est. speed input: 3806.06 toks/s, output: 918.17 toks/s]
Processed prompts:  95%|█████████▍| 9460/10000 [13:11<00:20, 26.32it/s, est. speed input: 3807.47 toks/s, output: 918.46 toks/s]
Processed prompts:  95%|█████████▍| 9464/10000 [13:11<00:23, 22.96it/s, est. speed input: 3808.49 toks/s, output: 918.60 toks/s]
Processed prompts:  95%|█████████▍| 9467/10000 [13:12<00:24, 21.76it/s, est. speed input: 3809.19 toks/s, output: 918.75 toks/s]
Processed prompts:  95%|█████████▍| 9470/10000 [13:12<00:27, 19.01it/s, est. speed input: 3809.76 toks/s, output: 918.78 toks/s]
Processed prompts:  95%|█████████▍| 9473/10000 [13:12<00:29, 17.81it/s, est. speed input: 3810.47 toks/s, output: 918.90 toks/s]
Processed prompts:  95%|█████████▍| 9476/10000 [13:12<00:26, 19.93it/s, est. speed input: 3811.72 toks/s, output: 919.13 toks/s]
Processed prompts:  95%|█████████▍| 9479/10000 [13:12<00:25, 20.71it/s, est. speed input: 3812.61 toks/s, output: 919.32 toks/s]
Processed prompts:  95%|█████████▍| 9482/10000 [13:12<00:24, 21.29it/s, est. speed input: 3813.51 toks/s, output: 919.49 toks/s]
Processed prompts:  95%|█████████▍| 9485/10000 [13:12<00:23, 21.71it/s, est. speed input: 3814.23 toks/s, output: 919.64 toks/s]
Processed prompts:  95%|█████████▍| 9489/10000 [13:13<00:21, 24.23it/s, est. speed input: 3815.79 toks/s, output: 919.96 toks/s]
Processed prompts:  95%|█████████▍| 9494/10000 [13:13<00:18, 26.94it/s, est. speed input: 3817.59 toks/s, output: 920.34 toks/s]
Processed prompts:  95%|█████████▍| 9497/10000 [13:13<00:22, 22.78it/s, est. speed input: 3818.38 toks/s, output: 920.49 toks/s]
Processed prompts:  95%|█████████▌| 9506/10000 [13:13<00:19, 25.53it/s, est. speed input: 3821.20 toks/s, output: 920.95 toks/s]
Processed prompts:  95%|█████████▌| 9509/10000 [13:13<00:21, 23.10it/s, est. speed input: 3822.04 toks/s, output: 921.10 toks/s]
Processed prompts:  95%|█████████▌| 9514/10000 [13:14<00:19, 24.84it/s, est. speed input: 3823.61 toks/s, output: 921.42 toks/s]
Processed prompts:  95%|█████████▌| 9517/10000 [13:14<00:18, 25.70it/s, est. speed input: 3824.77 toks/s, output: 921.67 toks/s]
Processed prompts:  95%|█████████▌| 9525/10000 [13:14<00:16, 29.18it/s, est. speed input: 3827.93 toks/s, output: 922.21 toks/s]
Processed prompts:  95%|█████████▌| 9528/10000 [13:14<00:16, 28.03it/s, est. speed input: 3829.04 toks/s, output: 922.48 toks/s]
Processed prompts:  95%|█████████▌| 9535/10000 [13:14<00:14, 31.08it/s, est. speed input: 3831.77 toks/s, output: 923.08 toks/s]
Processed prompts:  95%|█████████▌| 9539/10000 [13:14<00:17, 25.77it/s, est. speed input: 3832.67 toks/s, output: 923.29 toks/s]
Processed prompts:  96%|█████████▌| 9551/10000 [13:15<00:12, 35.69it/s, est. speed input: 3838.31 toks/s, output: 924.27 toks/s]
Processed prompts:  96%|█████████▌| 9555/10000 [13:15<00:14, 31.58it/s, est. speed input: 3839.50 toks/s, output: 924.43 toks/s]
Processed prompts:  96%|█████████▌| 9559/10000 [13:15<00:17, 24.83it/s, est. speed input: 3840.12 toks/s, output: 924.54 toks/s]
Processed prompts:  96%|█████████▌| 9566/10000 [13:15<00:14, 30.52it/s, est. speed input: 3842.98 toks/s, output: 925.14 toks/s]
Processed prompts:  96%|█████████▌| 9571/10000 [13:15<00:13, 30.84it/s, est. speed input: 3844.87 toks/s, output: 925.46 toks/s]
Processed prompts:  96%|█████████▌| 9576/10000 [13:16<00:16, 25.67it/s, est. speed input: 3846.32 toks/s, output: 925.63 toks/s]
Processed prompts:  96%|█████████▌| 9579/10000 [13:16<00:18, 23.31it/s, est. speed input: 3846.86 toks/s, output: 925.74 toks/s]
Processed prompts:  96%|█████████▌| 9588/10000 [13:16<00:13, 30.05it/s, est. speed input: 3850.26 toks/s, output: 926.48 toks/s]
Processed prompts:  96%|█████████▌| 9592/10000 [13:16<00:13, 29.83it/s, est. speed input: 3851.51 toks/s, output: 926.73 toks/s]
Processed prompts:  96%|█████████▌| 9596/10000 [13:16<00:13, 31.01it/s, est. speed input: 3853.08 toks/s, output: 926.95 toks/s]
Processed prompts:  96%|█████████▌| 9600/10000 [13:17<00:15, 26.62it/s, est. speed input: 3854.23 toks/s, output: 927.12 toks/s]
Processed prompts:  96%|█████████▌| 9609/10000 [13:17<00:12, 30.98it/s, est. speed input: 3857.76 toks/s, output: 927.82 toks/s]
Processed prompts:  96%|█████████▌| 9613/10000 [13:17<00:12, 31.24it/s, est. speed input: 3858.95 toks/s, output: 928.04 toks/s]
Processed prompts:  96%|█████████▌| 9617/10000 [13:17<00:15, 25.41it/s, est. speed input: 3859.74 toks/s, output: 928.17 toks/s]
Processed prompts:  96%|█████████▌| 9623/10000 [13:17<00:14, 25.64it/s, est. speed input: 3862.01 toks/s, output: 928.50 toks/s]
Processed prompts:  96%|█████████▋| 9629/10000 [13:18<00:12, 28.58it/s, est. speed input: 3864.39 toks/s, output: 929.00 toks/s]
Processed prompts:  96%|█████████▋| 9635/10000 [13:18<00:10, 34.02it/s, est. speed input: 3867.10 toks/s, output: 929.52 toks/s]
Processed prompts:  96%|█████████▋| 9641/10000 [13:18<00:10, 33.94it/s, est. speed input: 3869.14 toks/s, output: 929.86 toks/s]
Processed prompts:  96%|█████████▋| 9645/10000 [13:18<00:11, 31.82it/s, est. speed input: 3870.45 toks/s, output: 930.12 toks/s]
Processed prompts:  96%|█████████▋| 9649/10000 [13:18<00:12, 28.39it/s, est. speed input: 3871.75 toks/s, output: 930.38 toks/s]
Processed prompts:  97%|█████████▋| 9653/10000 [13:18<00:13, 25.86it/s, est. speed input: 3873.08 toks/s, output: 930.63 toks/s]
Processed prompts:  97%|█████████▋| 9656/10000 [13:18<00:13, 26.13it/s, est. speed input: 3874.10 toks/s, output: 930.82 toks/s]
Processed prompts:  97%|█████████▋| 9662/10000 [13:19<00:11, 28.45it/s, est. speed input: 3876.39 toks/s, output: 931.23 toks/s]
Processed prompts:  97%|█████████▋| 9665/10000 [13:19<00:12, 27.53it/s, est. speed input: 3877.31 toks/s, output: 931.39 toks/s]
Processed prompts:  97%|█████████▋| 9673/10000 [13:19<00:09, 32.83it/s, est. speed input: 3880.23 toks/s, output: 931.95 toks/s]
Processed prompts:  97%|█████████▋| 9677/10000 [13:19<00:10, 30.10it/s, est. speed input: 3881.33 toks/s, output: 932.15 toks/s]
Processed prompts:  97%|█████████▋| 9680/10000 [13:19<00:10, 30.04it/s, est. speed input: 3882.25 toks/s, output: 932.33 toks/s]
Processed prompts:  97%|█████████▋| 9683/10000 [13:19<00:11, 28.68it/s, est. speed input: 3883.11 toks/s, output: 932.48 toks/s]
Processed prompts:  97%|█████████▋| 9686/10000 [13:19<00:11, 27.39it/s, est. speed input: 3884.02 toks/s, output: 932.65 toks/s]
Processed prompts:  97%|█████████▋| 9689/10000 [13:20<00:11, 27.09it/s, est. speed input: 3885.16 toks/s, output: 932.86 toks/s]
Processed prompts:  97%|█████████▋| 9692/10000 [13:20<00:14, 21.21it/s, est. speed input: 3885.64 toks/s, output: 932.95 toks/s]
Processed prompts:  97%|█████████▋| 9696/10000 [13:20<00:14, 21.43it/s, est. speed input: 3886.84 toks/s, output: 933.19 toks/s]
Processed prompts:  97%|█████████▋| 9699/10000 [13:20<00:13, 23.04it/s, est. speed input: 3887.97 toks/s, output: 933.45 toks/s]
Processed prompts:  97%|█████████▋| 9702/10000 [13:20<00:17, 16.73it/s, est. speed input: 3887.98 toks/s, output: 933.40 toks/s]
Processed prompts:  97%|█████████▋| 9705/10000 [13:21<00:19, 15.39it/s, est. speed input: 3888.36 toks/s, output: 933.46 toks/s]
Processed prompts:  97%|█████████▋| 9707/10000 [13:21<00:23, 12.60it/s, est. speed input: 3888.04 toks/s, output: 933.37 toks/s]
Processed prompts:  97%|█████████▋| 9709/10000 [13:21<00:24, 12.06it/s, est. speed input: 3888.05 toks/s, output: 933.38 toks/s]
Processed prompts:  97%|█████████▋| 9712/10000 [13:21<00:21, 13.11it/s, est. speed input: 3888.64 toks/s, output: 933.51 toks/s]
Processed prompts:  97%|█████████▋| 9715/10000 [13:22<00:22, 12.75it/s, est. speed input: 3889.14 toks/s, output: 933.56 toks/s]
Processed prompts:  97%|█████████▋| 9719/10000 [13:22<00:21, 13.26it/s, est. speed input: 3889.77 toks/s, output: 933.64 toks/s]
Processed prompts:  97%|█████████▋| 9722/10000 [13:22<00:23, 11.67it/s, est. speed input: 3889.84 toks/s, output: 933.59 toks/s]
Processed prompts:  97%|█████████▋| 9725/10000 [13:22<00:23, 11.47it/s, est. speed input: 3890.15 toks/s, output: 933.61 toks/s]
Processed prompts:  97%|█████████▋| 9728/10000 [13:23<00:23, 11.46it/s, est. speed input: 3890.31 toks/s, output: 933.61 toks/s]
Processed prompts:  97%|█████████▋| 9730/10000 [13:23<00:26, 10.38it/s, est. speed input: 3890.21 toks/s, output: 933.53 toks/s]
Processed prompts:  97%|█████████▋| 9732/10000 [13:23<00:27,  9.66it/s, est. speed input: 3890.15 toks/s, output: 933.48 toks/s]
Processed prompts:  97%|█████████▋| 9734/10000 [13:23<00:28,  9.20it/s, est. speed input: 3889.77 toks/s, output: 933.39 toks/s]
Processed prompts:  97%|█████████▋| 9738/10000 [13:24<00:22, 11.75it/s, est. speed input: 3890.75 toks/s, output: 933.58 toks/s]
Processed prompts:  97%|█████████▋| 9740/10000 [13:24<00:25, 10.06it/s, est. speed input: 3890.45 toks/s, output: 933.49 toks/s]
Processed prompts:  97%|█████████▋| 9742/10000 [13:24<00:33,  7.72it/s, est. speed input: 3889.07 toks/s, output: 933.19 toks/s]
Processed prompts:  97%|█████████▋| 9748/10000 [13:25<00:23, 10.61it/s, est. speed input: 3890.00 toks/s, output: 933.40 toks/s]
Processed prompts:  98%|█████████▊| 9751/10000 [13:25<00:20, 11.89it/s, est. speed input: 3890.68 toks/s, output: 933.55 toks/s]
Processed prompts:  98%|█████████▊| 9757/10000 [13:25<00:14, 16.52it/s, est. speed input: 3892.31 toks/s, output: 934.01 toks/s]
Processed prompts:  98%|█████████▊| 9763/10000 [13:25<00:11, 20.39it/s, est. speed input: 3893.95 toks/s, output: 934.45 toks/s]
Processed prompts:  98%|█████████▊| 9768/10000 [13:25<00:10, 22.19it/s, est. speed input: 3895.16 toks/s, output: 934.79 toks/s]
Processed prompts:  98%|█████████▊| 9773/10000 [13:26<00:09, 23.96it/s, est. speed input: 3896.05 toks/s, output: 935.13 toks/s]
Processed prompts:  98%|█████████▊| 9776/10000 [13:26<00:10, 21.91it/s, est. speed input: 3896.14 toks/s, output: 935.22 toks/s]
Processed prompts:  98%|█████████▊| 9781/10000 [13:26<00:09, 23.57it/s, est. speed input: 3897.17 toks/s, output: 935.55 toks/s]
Processed prompts:  98%|█████████▊| 9785/10000 [13:26<00:09, 23.19it/s, est. speed input: 3897.96 toks/s, output: 935.84 toks/s]
Processed prompts:  98%|█████████▊| 9791/10000 [13:26<00:07, 26.76it/s, est. speed input: 3898.83 toks/s, output: 936.31 toks/s]
Processed prompts:  98%|█████████▊| 9797/10000 [13:27<00:07, 28.98it/s, est. speed input: 3899.71 toks/s, output: 936.82 toks/s]
Processed prompts:  98%|█████████▊| 9805/10000 [13:27<00:05, 34.65it/s, est. speed input: 3901.70 toks/s, output: 937.44 toks/s]
Processed prompts:  98%|█████████▊| 9811/10000 [13:27<00:05, 34.99it/s, est. speed input: 3903.05 toks/s, output: 937.90 toks/s]
Processed prompts:  98%|█████████▊| 9817/10000 [13:27<00:05, 35.52it/s, est. speed input: 3904.31 toks/s, output: 938.38 toks/s]
Processed prompts:  98%|█████████▊| 9822/10000 [13:27<00:05, 34.46it/s, est. speed input: 3905.38 toks/s, output: 938.77 toks/s]
Processed prompts:  98%|█████████▊| 9829/10000 [13:27<00:04, 37.48it/s, est. speed input: 3906.98 toks/s, output: 939.38 toks/s]
Processed prompts:  98%|█████████▊| 9838/10000 [13:27<00:03, 43.84it/s, est. speed input: 3909.20 toks/s, output: 940.21 toks/s]
Processed prompts:  98%|█████████▊| 9843/10000 [13:28<00:03, 40.67it/s, est. speed input: 3910.21 toks/s, output: 940.62 toks/s]
Processed prompts:  98%|█████████▊| 9849/10000 [13:28<00:04, 35.10it/s, est. speed input: 3911.07 toks/s, output: 940.98 toks/s]
Processed prompts:  99%|█████████▊| 9854/10000 [13:28<00:04, 35.21it/s, est. speed input: 3912.16 toks/s, output: 941.39 toks/s]
Processed prompts:  99%|█████████▊| 9862/10000 [13:28<00:03, 40.28it/s, est. speed input: 3914.11 toks/s, output: 942.14 toks/s]
Processed prompts:  99%|█████████▊| 9871/10000 [13:28<00:02, 46.10it/s, est. speed input: 3916.29 toks/s, output: 943.00 toks/s]
Processed prompts:  99%|█████████▉| 9876/10000 [13:28<00:02, 44.20it/s, est. speed input: 3917.34 toks/s, output: 943.48 toks/s]
Processed prompts:  99%|█████████▉| 9881/10000 [13:29<00:03, 39.22it/s, est. speed input: 3918.23 toks/s, output: 943.80 toks/s]
Processed prompts:  99%|█████████▉| 9889/10000 [13:29<00:02, 46.68it/s, est. speed input: 3920.10 toks/s, output: 944.55 toks/s]
Processed prompts:  99%|█████████▉| 9898/10000 [13:29<00:01, 54.54it/s, est. speed input: 3922.67 toks/s, output: 945.46 toks/s]
Processed prompts:  99%|█████████▉| 9904/10000 [13:29<00:01, 49.57it/s, est. speed input: 3924.06 toks/s, output: 946.01 toks/s]
Processed prompts:  99%|█████████▉| 9910/10000 [13:29<00:01, 46.07it/s, est. speed input: 3925.19 toks/s, output: 946.52 toks/s]
Processed prompts:  99%|█████████▉| 9915/10000 [13:29<00:01, 46.14it/s, est. speed input: 3926.47 toks/s, output: 946.94 toks/s]
Processed prompts:  99%|█████████▉| 9922/10000 [13:29<00:01, 46.05it/s, est. speed input: 3928.41 toks/s, output: 947.62 toks/s]
Processed prompts:  99%|█████████▉| 9934/10000 [13:30<00:01, 58.23it/s, est. speed input: 3932.10 toks/s, output: 948.89 toks/s]
Processed prompts:  99%|█████████▉| 9940/10000 [13:30<00:01, 54.00it/s, est. speed input: 3933.71 toks/s, output: 949.41 toks/s]
Processed prompts:  99%|█████████▉| 9946/10000 [13:30<00:01, 47.32it/s, est. speed input: 3934.80 toks/s, output: 949.91 toks/s]
Processed prompts: 100%|█████████▉| 9955/10000 [13:30<00:00, 53.73it/s, est. speed input: 3937.56 toks/s, output: 950.80 toks/s]
Processed prompts: 100%|█████████▉| 9965/10000 [13:30<00:00, 60.81it/s, est. speed input: 3940.71 toks/s, output: 951.82 toks/s]
Processed prompts: 100%|█████████▉| 9977/10000 [13:30<00:00, 72.70it/s, est. speed input: 3944.88 toks/s, output: 953.08 toks/s]
Processed prompts: 100%|█████████▉| 9985/10000 [13:30<00:00, 65.73it/s, est. speed input: 3946.34 toks/s, output: 953.84 toks/s]
Processed prompts: 100%|█████████▉| 9992/10000 [13:31<00:00, 45.31it/s, est. speed input: 3947.49 toks/s, output: 954.36 toks/s]
Processed prompts: 100%|█████████▉| 9998/10000 [13:31<00:00, 42.72it/s, est. speed input: 3949.02 toks/s, output: 954.94 toks/s]
Processed prompts: 100%|██████████| 10000/10000 [13:31<00:00, 42.72it/s, est. speed input: 3947.51 toks/s, output: 954.65 toks/s]
Processed prompts: 100%|██████████| 10000/10000 [13:31<00:00, 12.32it/s, est. speed input: 3947.51 toks/s, output: 954.65 toks/s]
{"event": "done", "out": "runs/e3/notes.jsonl", "stats": {"ok": 9370, "gold_label_named": 544, "phrase": 86}}
(MainProcess pid=29398) INFO 11-06 15:07:57 [utils.py:628] [shutdown] Process manager: send sigterm to process EngineCore
(EngineCore pid=29757) INFO 11-06 15:07:57 [core.py:1525] [shutdown] EngineCore: trigger received signal=SIGTERM
(EngineCore pid=29757) INFO 11-06 15:07:57 [core.py:1712] [shutdown] EngineCore: start mode=abort timeout=0s
(EngineCore pid=29757) INFO 11-06 15:07:57 [core.py:1743] [shutdown] EngineCore: request processing complete; starting resource teardown
(EngineCore pid=29757) INFO 11-06 15:07:57 [core.py:1573] [shutdown] EngineCore: exiting busy loop
Xet Storage Details

Size:: 409 kB
Xet hash:: 1636ff27fa3d1c8fac9923b8da3db3787eeb69f38059d6c6be2b1c4b2311246d
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.