Image-Text-to-Text
Transformers
Safetensors
hy_v4_flash
text-generation
hunyuan
hy4-flash
Mixture of Experts
multimodal
instruct
fp8
quantized
conversational
Instructions to use tencent/Hy4-flash-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tencent/Hy4-flash-FP8 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="tencent/Hy4-flash-FP8") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("tencent/Hy4-flash-FP8", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use tencent/Hy4-flash-FP8 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "tencent/Hy4-flash-FP8" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Hy4-flash-FP8", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/tencent/Hy4-flash-FP8
- SGLang
How to use tencent/Hy4-flash-FP8 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "tencent/Hy4-flash-FP8" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Hy4-flash-FP8", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "tencent/Hy4-flash-FP8" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Hy4-flash-FP8", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use tencent/Hy4-flash-FP8 with Docker Model Runner:
docker model run hf.co/tencent/Hy4-flash-FP8
update
Browse files- .gitattributes +1 -0
- README.md +50 -0
- README_CN.md +50 -0
- pretrained-model-bpb.png +3 -0
.gitattributes
CHANGED
|
@@ -37,3 +37,4 @@ assets/benchmark.png filter=lfs diff=lfs merge=lfs -text
|
|
| 37 |
assets/dfly2.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
assets/performance-cost-open-benchmarks.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
assets/performance-cost-workindex.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 37 |
assets/dfly2.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
assets/performance-cost-open-benchmarks.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
assets/performance-cost-workindex.png filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
pretrained-model-bpb.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -46,6 +46,7 @@ tags:
|
|
| 46 |
## Table of Contents
|
| 47 |
|
| 48 |
- [Model Introduction](#model-introduction)
|
|
|
|
| 49 |
- [Practicality and Cost Efficiency](#practicality-and-cost-efficiency)
|
| 50 |
- [Optimization Approach](#optimization-approach)
|
| 51 |
- [Task Performance and Cost](#task-performance-and-cost)
|
|
@@ -131,6 +132,55 @@ Compared with Hy4, Hy4 Flash introduces three notable architectural changes:
|
|
| 131 |
| Vision Position Embedding Size | 128 × 128 |
|
| 132 |
| Visual Feature Merging | 2 x 2 |
|
| 133 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 134 |
## Practicality and Cost Efficiency
|
| 135 |
|
| 136 |
### Optimization Approach
|
|
|
|
| 46 |
## Table of Contents
|
| 47 |
|
| 48 |
- [Model Introduction](#model-introduction)
|
| 49 |
+
- [Pretrained model capability](#pretrained-model-capability)
|
| 50 |
- [Practicality and Cost Efficiency](#practicality-and-cost-efficiency)
|
| 51 |
- [Optimization Approach](#optimization-approach)
|
| 52 |
- [Task Performance and Cost](#task-performance-and-cost)
|
|
|
|
| 132 |
| Vision Position Embedding Size | 128 × 128 |
|
| 133 |
| Visual Feature Merging | 2 x 2 |
|
| 134 |
|
| 135 |
+
## Pretrained model capability
|
| 136 |
+
|
| 137 |
+
Bits-per-byte (BPB) on internal code & agent validation, lower is better:
|
| 138 |
+
|
| 139 |
+
<p align="center">
|
| 140 |
+
<a href="pretrained-model-bpb.png"><img src="pretrained-model-bpb.png" alt="Bits-per-byte on internal code and agent validation versus active parameters; lower is better" width="100%"/></a>
|
| 141 |
+
</p>
|
| 142 |
+
|
| 143 |
+
Few-shot pretrain benchmark evaluation:
|
| 144 |
+
|
| 145 |
+
| Category | Benchmark | #Shots | Hy4 flash Base | Hy4 Base | Hy3 preview Base | DeepSeek-V4-flash-Base | DeepSeek-V4-Pro-Base |
|
| 146 |
+
| --- | --- | :---: | ---: | ---: | ---: | ---: | ---: |
|
| 147 |
+
| **Knowledge** | MMLU† | 5 | 88.7 | **91.6** | 87.5 | 89.7 | 91.1 |
|
| 148 |
+
| | MMLU-Pro† | 5 | 71.1 | **73.9** | 65.8 | 68.6 | 73.4 |
|
| 149 |
+
| | MMLU-Redux† | 5 | 87.6 | **90.6** | 87.0 | 88.8 | 90.2 |
|
| 150 |
+
| | SuperGPQA† | 5 | 57.8 | **62.7** | 52.3 | 52.5 | 54.8 |
|
| 151 |
+
| | C-Eval† | 5 | 92.0 | **95.0** | 90.1 | 91.9 | 93.4 |
|
| 152 |
+
| | CMMLU† | 5 | 91.6 | **93.1** | 89.6 | 89.8 | 90.7 |
|
| 153 |
+
| | SimpleQA | 5 | 37.5 | **57.2** | 26.4 | 31.3 | 52.1 |
|
| 154 |
+
| | Chinese-SimpleQA† | 5 | 74.6 | **83.9** | 69.6 | 71.9 | 76.6 |
|
| 155 |
+
| | AGIEval (English)† | 3 | 75.4 | **79.3** | 76.0 | 76.3 | 78.2 |
|
| 156 |
+
| | AGIEval (Chinese)† | 3 | 90.6 | **91.2** | 87.8 | 88.5 | 89.8 |
|
| 157 |
+
| **Reasoning** | HellaSwag (3,000-item subset) | 10 | 85.2 | **87.1** | 85.7 | 85.1 | 86.8 |
|
| 158 |
+
| | PIQA | 4 | 84.4 | **85.3** | 84.6 | 83.8 | 84.9 |
|
| 159 |
+
| | ARC-Challenge | 0 | 95.0 | **97.3** | 96.0 | **97.3** | 97.0 |
|
| 160 |
+
| | Social IQa | 0 | 80.1 | **82.7** | 80.5 | 72.5 | 73.1 |
|
| 161 |
+
| | KOR-Bench | 3 | 54.7 | **57.6** | 54.6 | 53.4 | 55.4 |
|
| 162 |
+
| | C³ | 0 | 77.3 | **82.4** | 75.9 | 74.1 | 78.3 |
|
| 163 |
+
| **Math** | U-MATH | 5 | 69.8 | **71.6** | 60.9 | 57.4 | 64.8 |
|
| 164 |
+
| | OCW Courses | 5 | 64.0 | 66.7 | 62.6 | 63.3 | **68.2** |
|
| 165 |
+
| **Code** | MBPP-plus | 3 | 73.7 | 74.0 | 77.4 | 76.8 | **79.1** |
|
| 166 |
+
| | BigCodeBench (Full) | 0 | 58.8 | 47.9 | **59.6** | 58.0 | 57.5 |
|
| 167 |
+
| **Multilingual** | Belebele (5-language subset)† | 5 | 93.8 | 95.0 | 94.2 | 94.4 | **95.2** |
|
| 168 |
+
| | INCLUDE† | 5 | 82.8 | **88.1** | 84.3 | 84.0 | 84.3 |
|
| 169 |
+
| **Multimodal** | MMMU-Pro (Standard, 10 opts) | 4 | 52.0 | **59.6** | — | — | — |
|
| 170 |
+
| | SimpleVQA | 4 | 48.5 | **54.5** | — | — | — |
|
| 171 |
+
| | CVBench | 4 | 83.4 | **86.7** | — | — | — |
|
| 172 |
+
| | RealWorldQA | 4 | 71.1 | **72.3** | — | — | — |
|
| 173 |
+
| | TextVQA | 4 | 73.4 | **78.2** | — | — | — |
|
| 174 |
+
| | CountBenchQA | 4 | 86.6 | **90.6** | — | — | — |
|
| 175 |
+
|
| 176 |
+
† Macro-averaged accuracy for the marked text benchmarks. Other text rows use the accuracy metric reported by our evaluation suite. Bold marks the highest available numerical score in each row; it does not indicate statistical significance.
|
| 177 |
+
|
| 178 |
+
Multimodal results are reported only for Hy4 flash Base and Hy4 Base.
|
| 179 |
+
|
| 180 |
+
ARC-Challenge and HellaSwag use 299-item and 3,000-item subsets, respectively. INCLUDE uses a 5,000-example Lite subset covering 44 languages. Belebele covers German, Korean, Vietnamese, Spanish, and Russian (4,475 examples in total).
|
| 181 |
+
|
| 182 |
+
**Disclaimer: Because our base model was exposed to a large amount of synthetic instruction data during mid-training, it occasionally fails to follow the expected answer format in few-shot evaluations, which may distort the scores.**
|
| 183 |
+
|
| 184 |
## Practicality and Cost Efficiency
|
| 185 |
|
| 186 |
### Optimization Approach
|
README_CN.md
CHANGED
|
@@ -30,6 +30,7 @@
|
|
| 30 |
## 目录
|
| 31 |
|
| 32 |
- [模型介绍](#模型介绍)
|
|
|
|
| 33 |
- [实用性与成本效率](#实用性与成本效率)
|
| 34 |
- [优化思路](#优化思路)
|
| 35 |
- [任务效果与成本](#任务效果与成本)
|
|
@@ -115,6 +116,55 @@
|
|
| 115 |
| Vision Position Embedding Size | 128 × 128 |
|
| 116 |
| Visual Feature Merging | 2 x 2 |
|
| 117 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 118 |
## 实用性与成本效率
|
| 119 |
|
| 120 |
### 优化思路
|
|
|
|
| 30 |
## 目录
|
| 31 |
|
| 32 |
- [模型介绍](#模型介绍)
|
| 33 |
+
- [预训练模型能力](#预训练模型能力)
|
| 34 |
- [实用性与成本效率](#实用性与成本效率)
|
| 35 |
- [优化思路](#优化思路)
|
| 36 |
- [任务效果与成本](#任务效果与成本)
|
|
|
|
| 116 |
| Vision Position Embedding Size | 128 × 128 |
|
| 117 |
| Visual Feature Merging | 2 x 2 |
|
| 118 |
|
| 119 |
+
## 预训练模型能力
|
| 120 |
+
|
| 121 |
+
内部代码与 Agent 验证集上的 BPB(Bits-per-byte,每字节比特数),数值越低越好:
|
| 122 |
+
|
| 123 |
+
<p align="center">
|
| 124 |
+
<a href="pretrained-model-bpb.png"><img src="pretrained-model-bpb.png" alt="内部代码与 Agent 验证集上的 BPB 与激活参数量对比,BPB 越低越好" width="100%"/></a>
|
| 125 |
+
</p>
|
| 126 |
+
|
| 127 |
+
预训练模型的 few-shot 基准评测结果:
|
| 128 |
+
|
| 129 |
+
| 类别 | 评测基准 | #Shots | Hy4 flash Base | Hy4 Base | Hy3 preview Base | DeepSeek-V4-flash-Base | DeepSeek-V4-Pro-Base |
|
| 130 |
+
| --- | --- | :---: | ---: | ---: | ---: | ---: | ---: |
|
| 131 |
+
| **知识** | MMLU† | 5 | 88.7 | **91.6** | 87.5 | 89.7 | 91.1 |
|
| 132 |
+
| | MMLU-Pro† | 5 | 71.1 | **73.9** | 65.8 | 68.6 | 73.4 |
|
| 133 |
+
| | MMLU-Redux† | 5 | 87.6 | **90.6** | 87.0 | 88.8 | 90.2 |
|
| 134 |
+
| | SuperGPQA† | 5 | 57.8 | **62.7** | 52.3 | 52.5 | 54.8 |
|
| 135 |
+
| | C-Eval† | 5 | 92.0 | **95.0** | 90.1 | 91.9 | 93.4 |
|
| 136 |
+
| | CMMLU† | 5 | 91.6 | **93.1** | 89.6 | 89.8 | 90.7 |
|
| 137 |
+
| | SimpleQA | 5 | 37.5 | **57.2** | 26.4 | 31.3 | 52.1 |
|
| 138 |
+
| | Chinese-SimpleQA† | 5 | 74.6 | **83.9** | 69.6 | 71.9 | 76.6 |
|
| 139 |
+
| | AGIEval (English)† | 3 | 75.4 | **79.3** | 76.0 | 76.3 | 78.2 |
|
| 140 |
+
| | AGIEval (Chinese)† | 3 | 90.6 | **91.2** | 87.8 | 88.5 | 89.8 |
|
| 141 |
+
| **推理** | HellaSwag (3,000-item subset) | 10 | 85.2 | **87.1** | 85.7 | 85.1 | 86.8 |
|
| 142 |
+
| | PIQA | 4 | 84.4 | **85.3** | 84.6 | 83.8 | 84.9 |
|
| 143 |
+
| | ARC-Challenge | 0 | 95.0 | **97.3** | 96.0 | **97.3** | 97.0 |
|
| 144 |
+
| | Social IQa | 0 | 80.1 | **82.7** | 80.5 | 72.5 | 73.1 |
|
| 145 |
+
| | KOR-Bench | 3 | 54.7 | **57.6** | 54.6 | 53.4 | 55.4 |
|
| 146 |
+
| | C³ | 0 | 77.3 | **82.4** | 75.9 | 74.1 | 78.3 |
|
| 147 |
+
| **数学** | U-MATH | 5 | 69.8 | **71.6** | 60.9 | 57.4 | 64.8 |
|
| 148 |
+
| | OCW Courses | 5 | 64.0 | 66.7 | 62.6 | 63.3 | **68.2** |
|
| 149 |
+
| **代码** | MBPP-plus | 3 | 73.7 | 74.0 | 77.4 | 76.8 | **79.1** |
|
| 150 |
+
| | BigCodeBench (Full) | 0 | 58.8 | 47.9 | **59.6** | 58.0 | 57.5 |
|
| 151 |
+
| **多语言** | Belebele (5-language subset)† | 5 | 93.8 | 95.0 | 94.2 | 94.4 | **95.2** |
|
| 152 |
+
| | INCLUDE† | 5 | 82.8 | **88.1** | 84.3 | 84.0 | 84.3 |
|
| 153 |
+
| **多模态** | MMMU-Pro (Standard, 10 opts) | 4 | 52.0 | **59.6** | — | — | — |
|
| 154 |
+
| | SimpleVQA | 4 | 48.5 | **54.5** | — | — | — |
|
| 155 |
+
| | CVBench | 4 | 83.4 | **86.7** | — | — | — |
|
| 156 |
+
| | RealWorldQA | 4 | 71.1 | **72.3** | — | — | — |
|
| 157 |
+
| | TextVQA | 4 | 73.4 | **78.2** | — | — | — |
|
| 158 |
+
| | CountBenchQA | 4 | 86.6 | **90.6** | — | — | — |
|
| 159 |
+
|
| 160 |
+
† 标注的文本评测采用宏平均准确率,其余文本评测采用本评测框架报告的准确率。每行最高分以粗体标示;加粗不代表差异具有统计显著性。
|
| 161 |
+
|
| 162 |
+
多模态结果仅列出 Hy4 flash Base 和 Hy4 Base。
|
| 163 |
+
|
| 164 |
+
ARC-Challenge 和 HellaSwag 分别使用包含 299 道和 3,000 道题目的子集。INCLUDE 使用覆盖 44 种语言、包含 5,000 个样本的 Lite 子集。Belebele 覆盖德语、韩语、越南语、西班牙语和俄语,共 4,475 个样本。
|
| 165 |
+
|
| 166 |
+
**免责声明:我们的基座模型在中期训练中使用了大量合成指令数据,因此在 few-shot 评测中偶尔无法遵循预期的答案格式,可能导致得分失真。**
|
| 167 |
+
|
| 168 |
## 实用性与成本效率
|
| 169 |
|
| 170 |
### 优化思路
|
pretrained-model-bpb.png
ADDED
|
Git LFS Details
|