Qwen3.8-27B-Quant / validation /http_validation.log
DemoAuguste's picture
Add Qwen3.8-27B SLQ b3.6 fake-quant checkpoint (64 shards)
464c1c3 verified
Raw History Blame Contribute Delete
1.34 kB
=== 1) /health ===
status=200
=== 2) /v1/models ===
id= qwen3.8-27b-slq-b36 max_model_len= 4096 root= /work/b36
=== 3) /v1/chat/completions (non-stream) ===
content: 混合精度量化是一种在模型训练或推理过程中,同时使用多种数值精度(如 FP32 和 FP16/BF16)来平衡计算效率与数值稳定性的技术。
usage: {'prompt_tokens': 20, 'total_tokens': 62, 'completion_tokens': 42, 'prompt_tokens_details': None, 'completion_tokens_details': None}
wall: 13.848591920s
=== 4) /v1/completions ===
text: Beijing, which is also the largest city in the country. Beijing is a major political, cultural, and educational center,
usage: {'prompt_tokens': 5, 'total_tokens': 29, 'completion_tokens': 24, 'prompt_tokens_details': None, 'completion_tokens_details': None}
=== 5) streaming (SSE) ===
data: {"id":"chatcmpl-a139a13ce3a3826e","object":"chat.completion.chunk","created":1794997579,"model":"qwen3.8-27b-slq-b36","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}],"prompt_token_ids":null,"prompt_text":null}
data: {"id":"chatcmpl-a139a13ce3a3826e","object":"chat.completion.chunk","created":1794997579,"model":"qwen3.8-27b-slq-b36","choices":[{"index":0,"delta":{"content":"北京"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
=== SSE_OK ===