Image-Text-to-Text
Transformers
Safetensors
hy_v4_flash
text-generation
hunyuan
hy4-flash
Mixture of Experts
multimodal
instruct
fp8
quantized
conversational
Instructions to use tencent/Hy4-flash-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tencent/Hy4-flash-FP8 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="tencent/Hy4-flash-FP8") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("tencent/Hy4-flash-FP8", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use tencent/Hy4-flash-FP8 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "tencent/Hy4-flash-FP8" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Hy4-flash-FP8", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/tencent/Hy4-flash-FP8
- SGLang
How to use tencent/Hy4-flash-FP8 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "tencent/Hy4-flash-FP8" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Hy4-flash-FP8", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "tencent/Hy4-flash-FP8" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Hy4-flash-FP8", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use tencent/Hy4-flash-FP8 with Docker Model Runner:
docker model run hf.co/tencent/Hy4-flash-FP8
update
Browse files- README.md +1 -2
- README_CN.md +1 -2
README.md
CHANGED
|
@@ -137,7 +137,7 @@ Compared with Hy4, Hy4 Flash introduces three notable architectural changes:
|
|
| 137 |
Bits-per-byte (BPB) on internal code & agent validation, lower is better:
|
| 138 |
|
| 139 |
<p align="center">
|
| 140 |
-
<a href="pretrained-model-bpb.png"><img src="pretrained-model-bpb.png" alt="Bits-per-byte on internal code and agent validation versus active parameters; lower is better" width="
|
| 141 |
</p>
|
| 142 |
|
| 143 |
Few-shot pretrain benchmark evaluation:
|
|
@@ -163,7 +163,6 @@ Few-shot pretrain benchmark evaluation:
|
|
| 163 |
| **Math** | U-MATH | 5 | 69.8 | **71.6** | 60.9 | 57.4 | 64.8 |
|
| 164 |
| | OCW Courses | 5 | 64.0 | 66.7 | 62.6 | 63.3 | **68.2** |
|
| 165 |
| **Code** | MBPP-plus | 3 | 73.7 | 74.0 | 77.4 | 76.8 | **79.1** |
|
| 166 |
-
| | BigCodeBench (Full) | 0 | 58.8 | 47.9 | **59.6** | 58.0 | 57.5 |
|
| 167 |
| **Multilingual** | Belebele (5-language subset)† | 5 | 93.8 | 95.0 | 94.2 | 94.4 | **95.2** |
|
| 168 |
| | INCLUDE† | 5 | 82.8 | **88.1** | 84.3 | 84.0 | 84.3 |
|
| 169 |
| **Multimodal** | MMMU-Pro (Standard, 10 opts) | 4 | 52.0 | **59.6** | — | — | — |
|
|
|
|
| 137 |
Bits-per-byte (BPB) on internal code & agent validation, lower is better:
|
| 138 |
|
| 139 |
<p align="center">
|
| 140 |
+
<a href="pretrained-model-bpb.png"><img src="pretrained-model-bpb.png" alt="Bits-per-byte on internal code and agent validation versus active parameters; lower is better" width="75%"/></a>
|
| 141 |
</p>
|
| 142 |
|
| 143 |
Few-shot pretrain benchmark evaluation:
|
|
|
|
| 163 |
| **Math** | U-MATH | 5 | 69.8 | **71.6** | 60.9 | 57.4 | 64.8 |
|
| 164 |
| | OCW Courses | 5 | 64.0 | 66.7 | 62.6 | 63.3 | **68.2** |
|
| 165 |
| **Code** | MBPP-plus | 3 | 73.7 | 74.0 | 77.4 | 76.8 | **79.1** |
|
|
|
|
| 166 |
| **Multilingual** | Belebele (5-language subset)† | 5 | 93.8 | 95.0 | 94.2 | 94.4 | **95.2** |
|
| 167 |
| | INCLUDE† | 5 | 82.8 | **88.1** | 84.3 | 84.0 | 84.3 |
|
| 168 |
| **Multimodal** | MMMU-Pro (Standard, 10 opts) | 4 | 52.0 | **59.6** | — | — | — |
|
README_CN.md
CHANGED
|
@@ -121,7 +121,7 @@
|
|
| 121 |
内部代码与 Agent 验证集上的 BPB(Bits-per-byte,每字节比特数),数值越低越好:
|
| 122 |
|
| 123 |
<p align="center">
|
| 124 |
-
<a href="pretrained-model-bpb.png"><img src="pretrained-model-bpb.png" alt="内部代码与 Agent 验证集上的 BPB 与激活参数量对比,BPB 越低越好" width="
|
| 125 |
</p>
|
| 126 |
|
| 127 |
预训练模型的 few-shot 基准评测结果:
|
|
@@ -147,7 +147,6 @@
|
|
| 147 |
| **数学** | U-MATH | 5 | 69.8 | **71.6** | 60.9 | 57.4 | 64.8 |
|
| 148 |
| | OCW Courses | 5 | 64.0 | 66.7 | 62.6 | 63.3 | **68.2** |
|
| 149 |
| **代码** | MBPP-plus | 3 | 73.7 | 74.0 | 77.4 | 76.8 | **79.1** |
|
| 150 |
-
| | BigCodeBench (Full) | 0 | 58.8 | 47.9 | **59.6** | 58.0 | 57.5 |
|
| 151 |
| **多语言** | Belebele (5-language subset)† | 5 | 93.8 | 95.0 | 94.2 | 94.4 | **95.2** |
|
| 152 |
| | INCLUDE† | 5 | 82.8 | **88.1** | 84.3 | 84.0 | 84.3 |
|
| 153 |
| **多模态** | MMMU-Pro (Standard, 10 opts) | 4 | 52.0 | **59.6** | — | — | — |
|
|
|
|
| 121 |
内部代码与 Agent 验证集上的 BPB(Bits-per-byte,每字节比特数),数值越低越好:
|
| 122 |
|
| 123 |
<p align="center">
|
| 124 |
+
<a href="pretrained-model-bpb.png"><img src="pretrained-model-bpb.png" alt="内部代码与 Agent 验证集上的 BPB 与激活参数量对比,BPB 越低越好" width="75%"/></a>
|
| 125 |
</p>
|
| 126 |
|
| 127 |
预训练模型的 few-shot 基准评测结果:
|
|
|
|
| 147 |
| **数学** | U-MATH | 5 | 69.8 | **71.6** | 60.9 | 57.4 | 64.8 |
|
| 148 |
| | OCW Courses | 5 | 64.0 | 66.7 | 62.6 | 63.3 | **68.2** |
|
| 149 |
| **代码** | MBPP-plus | 3 | 73.7 | 74.0 | 77.4 | 76.8 | **79.1** |
|
|
|
|
| 150 |
| **多语言** | Belebele (5-language subset)† | 5 | 93.8 | 95.0 | 94.2 | 94.4 | **95.2** |
|
| 151 |
| | INCLUDE† | 5 | 82.8 | **88.1** | 84.3 | 84.0 | 84.3 |
|
| 152 |
| **多模态** | MMMU-Pro (Standard, 10 opts) | 4 | 52.0 | **59.6** | — | — | — |
|