Diffusers
Safetensors
PyTorch
hy-nomas
world-model
video-generation
interactive-video-generation
camera-control
autoregressive
diffusion-transformer
Instructions to use tencent/Hy-Nomas with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use tencent/Hy-Nomas with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("tencent/Hy-Nomas", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Commit ·
bac719a
1
Parent(s): 5cdaa8c
Update English and Chinese model cards
Browse files- README.md +16 -11
- README_CN.md +15 -11
README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
-
|
| 4 |
tags:
|
| 5 |
- hy-nomas
|
| 6 |
- world-model
|
|
@@ -13,7 +13,7 @@ tags:
|
|
| 13 |
---
|
| 14 |
|
| 15 |
<p align="left">
|
| 16 |
-
<a href="README_CN.md">中文</a> | English
|
| 17 |
</p>
|
| 18 |
|
| 19 |
<p align="center">
|
|
@@ -26,13 +26,18 @@ tags:
|
|
| 26 |
<a href="https://github.com/Tencent-Hunyuan/Hy-Nomas"><img src="https://img.shields.io/badge/GitHub-181717?style=flat-square&logo=github&logoColor=white" alt="GitHub"/></a>
|
| 27 |
<a href="https://github.com/Tencent-Hunyuan/Hy-Nomas/blob/main/Hy_Nomas_Tech_Report.pdf"><img src="https://img.shields.io/badge/Technical%20Report%20%28Coming%20Soon%29-B31B1B?style=flat-square&logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSJ3aGl0ZSIgc3Ryb2tlLXdpZHRoPSIyIiBzdHJva2UtbGluZWpvaW49InJvdW5kIj48cGF0aCBkPSJNMTQgMkg1djIwaDE0Vjd6TTE0IDJ2NWg1TTggMTJoOE04IDE2aDgiLz48L3N2Zz4%3D" alt="Technical Report (Coming Soon)"/></a>
|
| 28 |
<a href="https://huggingface.co/datasets/tencent/Hy-Nomas-Bench"><img src="https://img.shields.io/badge/Benchmark-FFD21E?style=flat-square&logo=huggingface&logoColor=black" alt="Benchmark"/></a>
|
| 29 |
-
<a href="https://hy-nomas.hy.tencent.com/"><img src="https://img.shields.io/badge/
|
|
|
|
| 30 |
</p>
|
| 31 |
|
| 32 |
<h3 align="center">
|
| 33 |
🚀 <a href="https://hy-nomas.hy.tencent.com/">Try the Demo →</a>
|
| 34 |
</h3>
|
| 35 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
## Model Introduction
|
| 37 |
|
| 38 |
**Hy-Nomas** is a video world generation model that supports free-form, multi-turn interaction through text instructions and camera control. It generates a world one video segment at a time, allowing users to introduce objects, change styles and scenes, describe actions, or control the camera as generation continues.
|
|
@@ -175,7 +180,7 @@ Run the commands below from the Hy-Nomas **code repository root**. They use this
|
|
| 175 |
|
| 176 |
### Prepare Prompts
|
| 177 |
|
| 178 |
-
Offline inference accepts a JSON list of video-generation requests. Each request can contain multiple `segments`
|
| 179 |
|
| 180 |
The inference commands below use the bundled [example prompt file](examples/prompts.json) in the model repository. For custom content, save a JSON file in the following format and set `inference.prompt=/path/to/prompts.json`:
|
| 181 |
|
|
@@ -194,19 +199,19 @@ The inference commands below use the bundled [example prompt file](examples/prom
|
|
| 194 |
],
|
| 195 |
"segments": [
|
| 196 |
{
|
| 197 |
-
"start_time":
|
| 198 |
-
"end_time":
|
| 199 |
"visual_description_long": "A small orange fox stands on a stone path in a luminous forest. Blue fireflies drift between the trees.",
|
| 200 |
"contained_actions": [
|
| 201 |
-
{"start_time":
|
| 202 |
]
|
| 203 |
},
|
| 204 |
{
|
| 205 |
-
"start_time":
|
| 206 |
-
"end_time":
|
| 207 |
"description_change": "Glowing blue flowers open beside the path as the fox continues forward.",
|
| 208 |
"contained_actions": [
|
| 209 |
-
{"start_time":
|
| 210 |
]
|
| 211 |
}
|
| 212 |
]
|
|
@@ -277,7 +282,7 @@ For local deployment, follow the [web demo deployment guide](https://github.com/
|
|
| 277 |
|
| 278 |
## License
|
| 279 |
|
| 280 |
-
Hy-Nomas is licensed under
|
| 281 |
|
| 282 |
## Acknowledgement
|
| 283 |
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
license_link: https://huggingface.co/tencent/Hy-Nomas/blob/main/LICENSE
|
| 4 |
tags:
|
| 5 |
- hy-nomas
|
| 6 |
- world-model
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
<p align="left">
|
| 16 |
+
<a href="https://huggingface.co/tencent/Hy-Nomas/blob/main/README_CN.md">中文</a> | English
|
| 17 |
</p>
|
| 18 |
|
| 19 |
<p align="center">
|
|
|
|
| 26 |
<a href="https://github.com/Tencent-Hunyuan/Hy-Nomas"><img src="https://img.shields.io/badge/GitHub-181717?style=flat-square&logo=github&logoColor=white" alt="GitHub"/></a>
|
| 27 |
<a href="https://github.com/Tencent-Hunyuan/Hy-Nomas/blob/main/Hy_Nomas_Tech_Report.pdf"><img src="https://img.shields.io/badge/Technical%20Report%20%28Coming%20Soon%29-B31B1B?style=flat-square&logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSJ3aGl0ZSIgc3Ryb2tlLXdpZHRoPSIyIiBzdHJva2UtbGluZWpvaW49InJvdW5kIj48cGF0aCBkPSJNMTQgMkg1djIwaDE0Vjd6TTE0IDJ2NWg1TTggMTJoOE04IDE2aDgiLz48L3N2Zz4%3D" alt="Technical Report (Coming Soon)"/></a>
|
| 28 |
<a href="https://huggingface.co/datasets/tencent/Hy-Nomas-Bench"><img src="https://img.shields.io/badge/Benchmark-FFD21E?style=flat-square&logo=huggingface&logoColor=black" alt="Benchmark"/></a>
|
| 29 |
+
<a href="https://hy-nomas.hy.tencent.com/"><img src="https://img.shields.io/badge/Project%20Website-666666?style=flat-square&logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSJ3aGl0ZSIgc3Ryb2tlLXdpZHRoPSIyIj48Y2lyY2xlIGN4PSIxMiIgY3k9IjEyIiByPSIxMCIvPjxlbGxpcHNlIGN4PSIxMiIgY3k9IjEyIiByeD0iNCIgcnk9IjEwIi8%2BPHBhdGggZD0iTTIgMTJoMjAiLz48L3N2Zz4%3D" alt="Project Website"/></a>
|
| 30 |
+
<a href="https://x.com/TencentHunyuan/status/2113676827114053892"><img src="https://img.shields.io/badge/X%20Post-000000?style=flat-square&logo=x&logoColor=white" alt="X Post"/></a>
|
| 31 |
</p>
|
| 32 |
|
| 33 |
<h3 align="center">
|
| 34 |
🚀 <a href="https://hy-nomas.hy.tencent.com/">Try the Demo →</a>
|
| 35 |
</h3>
|
| 36 |
|
| 37 |
+
<br>
|
| 38 |
+
|
| 39 |
+
<video controls playsinline preload="metadata" width="100%" src="https://hy-nomas.hy.tencent.com/gallery/hy_nomas_promo.mp4"></video>
|
| 40 |
+
|
| 41 |
## Model Introduction
|
| 42 |
|
| 43 |
**Hy-Nomas** is a video world generation model that supports free-form, multi-turn interaction through text instructions and camera control. It generates a world one video segment at a time, allowing users to introduce objects, change styles and scenes, describe actions, or control the camera as generation continues.
|
|
|
|
| 180 |
|
| 181 |
### Prepare Prompts
|
| 182 |
|
| 183 |
+
Offline inference accepts a JSON list of video-generation requests. Each request can contain multiple `segments`. Both segments and their `contained_actions` use `start_time` and `end_time` strings in `MM:SS.mmm` format (minutes:seconds.milliseconds). The example below contains two 5-second segments covering 0–5 seconds and 5–10 seconds. The first segment describes the initial world, and later segments use `description_change` to describe how it should evolve.
|
| 184 |
|
| 185 |
The inference commands below use the bundled [example prompt file](examples/prompts.json) in the model repository. For custom content, save a JSON file in the following format and set `inference.prompt=/path/to/prompts.json`:
|
| 186 |
|
|
|
|
| 199 |
],
|
| 200 |
"segments": [
|
| 201 |
{
|
| 202 |
+
"start_time": "00:00.000",
|
| 203 |
+
"end_time": "00:05.000",
|
| 204 |
"visual_description_long": "A small orange fox stands on a stone path in a luminous forest. Blue fireflies drift between the trees.",
|
| 205 |
"contained_actions": [
|
| 206 |
+
{"start_time": "00:00.000", "end_time": "00:05.000", "action_description": "The fox walks slowly along the path."}
|
| 207 |
]
|
| 208 |
},
|
| 209 |
{
|
| 210 |
+
"start_time": "00:05.000",
|
| 211 |
+
"end_time": "00:10.000",
|
| 212 |
"description_change": "Glowing blue flowers open beside the path as the fox continues forward.",
|
| 213 |
"contained_actions": [
|
| 214 |
+
{"start_time": "00:05.000", "end_time": "00:10.000", "action_description": "The fox passes the flowers and looks around."}
|
| 215 |
]
|
| 216 |
}
|
| 217 |
]
|
|
|
|
| 282 |
|
| 283 |
## License
|
| 284 |
|
| 285 |
+
Hy-Nomas is licensed under Apache-2.0. Please refer to this repository's [LICENSE](https://huggingface.co/tencent/Hy-Nomas/blob/main/LICENSE) file for the applicable terms. Third-party models and components remain subject to their respective licenses.
|
| 286 |
|
| 287 |
## Acknowledgement
|
| 288 |
|
README_CN.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
-
|
| 4 |
tags:
|
| 5 |
- hy-nomas
|
| 6 |
- world-model
|
|
@@ -13,7 +13,7 @@ tags:
|
|
| 13 |
---
|
| 14 |
|
| 15 |
<p align="left">
|
| 16 |
-
<a href="
|
| 17 |
</p>
|
| 18 |
|
| 19 |
<p align="center">
|
|
@@ -26,13 +26,17 @@ tags:
|
|
| 26 |
<a href="https://github.com/Tencent-Hunyuan/Hy-Nomas"><img src="https://img.shields.io/badge/GitHub-181717?style=flat-square&logo=github&logoColor=white" alt="GitHub"/></a>
|
| 27 |
<a href="https://github.com/Tencent-Hunyuan/Hy-Nomas/blob/main/Hy_Nomas_Tech_Report.pdf"><img src="https://img.shields.io/badge/%E6%8A%80%E6%9C%AF%E6%8A%A5%E5%91%8A%20%28Coming%20Soon%29-B31B1B?style=flat-square&logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSJ3aGl0ZSIgc3Ryb2tlLXdpZHRoPSIyIiBzdHJva2UtbGluZWpvaW49InJvdW5kIj48cGF0aCBkPSJNMTQgMkg1djIwaDE0Vjd6TTE0IDJ2NWg1TTggMTJoOE04IDE2aDgiLz48L3N2Zz4%3D" alt="技术报告 (Coming Soon)"/></a>
|
| 28 |
<a href="https://huggingface.co/datasets/tencent/Hy-Nomas-Bench"><img src="https://img.shields.io/badge/%E8%AF%84%E6%B5%8B%E5%9F%BA%E5%87%86-FFD21E?style=flat-square&logo=huggingface&logoColor=black" alt="评测基准"/></a>
|
| 29 |
-
<a href="https://hy-nomas.hy.tencent.com/"><img src="https://img.shields.io/badge/
|
|
|
|
| 30 |
</p>
|
| 31 |
|
| 32 |
<h3 align="center">
|
| 33 |
🚀 <a href="https://hy-nomas.hy.tencent.com/">体验演示 →</a>
|
| 34 |
</h3>
|
| 35 |
|
|
|
|
|
|
|
|
|
|
| 36 |
## 模型介绍
|
| 37 |
|
| 38 |
**Hy-Nomas** 是一个支持自由形式、多轮交互的视频世界生成模型,可通过文本指令和相机控制信号逐段生成视频世界。用户可以在生成过程中引入物体、改变风格和场景、描述动作,或控制相机运动。
|
|
@@ -175,7 +179,7 @@ export HY_NOMAS_MODEL_DIR="/path/to/Hy-Nomas-models"
|
|
| 175 |
|
| 176 |
### 准备提示词
|
| 177 |
|
| 178 |
-
离线推理支持包含多个视频生成请求的 JSON 列表。每个请求可包含多个 `segments`
|
| 179 |
|
| 180 |
下方推理命令使用模型仓库中随附的[提示词示例文件](examples/prompts.json)。如需自定义内容,可按以下格式保存 JSON,并通过 `inference.prompt=/path/to/prompts.json` 指定文件:
|
| 181 |
|
|
@@ -194,19 +198,19 @@ export HY_NOMAS_MODEL_DIR="/path/to/Hy-Nomas-models"
|
|
| 194 |
],
|
| 195 |
"segments": [
|
| 196 |
{
|
| 197 |
-
"start_time":
|
| 198 |
-
"end_time":
|
| 199 |
"visual_description_long": "A small orange fox stands on a stone path in a luminous forest. Blue fireflies drift between the trees.",
|
| 200 |
"contained_actions": [
|
| 201 |
-
{"start_time":
|
| 202 |
]
|
| 203 |
},
|
| 204 |
{
|
| 205 |
-
"start_time":
|
| 206 |
-
"end_time":
|
| 207 |
"description_change": "Glowing blue flowers open beside the path as the fox continues forward.",
|
| 208 |
"contained_actions": [
|
| 209 |
-
{"start_time":
|
| 210 |
]
|
| 211 |
}
|
| 212 |
]
|
|
@@ -277,7 +281,7 @@ bash scripts/inference/inference_from_config.sh configs/experiment/single_gpu_lo
|
|
| 277 |
|
| 278 |
## 许可证
|
| 279 |
|
| 280 |
-
Hy-Nomas 采用
|
| 281 |
|
| 282 |
## 致谢
|
| 283 |
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
license_link: https://huggingface.co/tencent/Hy-Nomas/blob/main/LICENSE
|
| 4 |
tags:
|
| 5 |
- hy-nomas
|
| 6 |
- world-model
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
<p align="left">
|
| 16 |
+
<a href="https://huggingface.co/tencent/Hy-Nomas">English</a> | 中文
|
| 17 |
</p>
|
| 18 |
|
| 19 |
<p align="center">
|
|
|
|
| 26 |
<a href="https://github.com/Tencent-Hunyuan/Hy-Nomas"><img src="https://img.shields.io/badge/GitHub-181717?style=flat-square&logo=github&logoColor=white" alt="GitHub"/></a>
|
| 27 |
<a href="https://github.com/Tencent-Hunyuan/Hy-Nomas/blob/main/Hy_Nomas_Tech_Report.pdf"><img src="https://img.shields.io/badge/%E6%8A%80%E6%9C%AF%E6%8A%A5%E5%91%8A%20%28Coming%20Soon%29-B31B1B?style=flat-square&logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSJ3aGl0ZSIgc3Ryb2tlLXdpZHRoPSIyIiBzdHJva2UtbGluZWpvaW49InJvdW5kIj48cGF0aCBkPSJNMTQgMkg1djIwaDE0Vjd6TTE0IDJ2NWg1TTggMTJoOE04IDE2aDgiLz48L3N2Zz4%3D" alt="技术报告 (Coming Soon)"/></a>
|
| 28 |
<a href="https://huggingface.co/datasets/tencent/Hy-Nomas-Bench"><img src="https://img.shields.io/badge/%E8%AF%84%E6%B5%8B%E5%9F%BA%E5%87%86-FFD21E?style=flat-square&logo=huggingface&logoColor=black" alt="评测基准"/></a>
|
| 29 |
+
<a href="https://hy-nomas.hy.tencent.com/"><img src="https://img.shields.io/badge/Project%20Website-666666?style=flat-square&logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSJ3aGl0ZSIgc3Ryb2tlLXdpZHRoPSIyIj48Y2lyY2xlIGN4PSIxMiIgY3k9IjEyIiByPSIxMCIvPjxlbGxpcHNlIGN4PSIxMiIgY3k9IjEyIiByeD0iNCIgcnk9IjEwIi8%2BPHBhdGggZD0iTTIgMTJoMjAiLz48L3N2Zz4%3D" alt="Project Website"/></a>
|
| 30 |
+
<a href="https://x.com/TencentHunyuan/status/2113676827114053892"><img src="https://img.shields.io/badge/X%20Post-000000?style=flat-square&logo=x&logoColor=white" alt="X Post"/></a>
|
| 31 |
</p>
|
| 32 |
|
| 33 |
<h3 align="center">
|
| 34 |
🚀 <a href="https://hy-nomas.hy.tencent.com/">体验演示 →</a>
|
| 35 |
</h3>
|
| 36 |
|
| 37 |
+
|
| 38 |
+
<video controls playsinline preload="metadata" width="100%" src="https://hy-nomas.hy.tencent.com/gallery/hy_nomas_promo.mp4"></video>
|
| 39 |
+
|
| 40 |
## 模型介绍
|
| 41 |
|
| 42 |
**Hy-Nomas** 是一个支持自由形式、多轮交互的视频世界生成模型,可通过文本指令和相机控制信号逐段生成视频世界。用户可以在生成过程中引入物体、改变风格和场景、描述动作,或控制相机运动。
|
|
|
|
| 179 |
|
| 180 |
### 准备提示词
|
| 181 |
|
| 182 |
+
离线推理支持包含多个视频生成请求的 JSON 列表。每个请求可包含多个 `segments`。片段及其 `contained_actions` 中的 `start_time` 和 `end_time` 均使用 `MM:SS.mmm`(分钟:秒.毫秒)格式的字符串。下方示例包含两个各 5 秒的片段,时间范围分别为 0–5 秒和 5–10 秒。首个片段描述初始世界,后续片段通过 `description_change` 描述世界如何变化。
|
| 183 |
|
| 184 |
下方推理命令使用模型仓库中随附的[提示词示例文件](examples/prompts.json)。如需自定义内容,可按以下格式保存 JSON,并通过 `inference.prompt=/path/to/prompts.json` 指定文件:
|
| 185 |
|
|
|
|
| 198 |
],
|
| 199 |
"segments": [
|
| 200 |
{
|
| 201 |
+
"start_time": "00:00.000",
|
| 202 |
+
"end_time": "00:05.000",
|
| 203 |
"visual_description_long": "A small orange fox stands on a stone path in a luminous forest. Blue fireflies drift between the trees.",
|
| 204 |
"contained_actions": [
|
| 205 |
+
{"start_time": "00:00.000", "end_time": "00:05.000", "action_description": "The fox walks slowly along the path."}
|
| 206 |
]
|
| 207 |
},
|
| 208 |
{
|
| 209 |
+
"start_time": "00:05.000",
|
| 210 |
+
"end_time": "00:10.000",
|
| 211 |
"description_change": "Glowing blue flowers open beside the path as the fox continues forward.",
|
| 212 |
"contained_actions": [
|
| 213 |
+
{"start_time": "00:05.000", "end_time": "00:10.000", "action_description": "The fox passes the flowers and looks around."}
|
| 214 |
]
|
| 215 |
}
|
| 216 |
]
|
|
|
|
| 281 |
|
| 282 |
## 许可证
|
| 283 |
|
| 284 |
+
Hy-Nomas 采用 Apache-2.0 许可证,适用条款请参阅本仓库的 [LICENSE](https://huggingface.co/tencent/Hy-Nomas/blob/main/LICENSE) 文件。第三方模型和组件仍遵循各自的许可证。
|
| 285 |
|
| 286 |
## 致谢
|
| 287 |
|