XD-MU commited on
Commit
31acbfa
·
verified ·
1 Parent(s): c2352c3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +29 -30
README.md CHANGED
@@ -2,59 +2,58 @@
2
  license: other
3
  library_name: transformers
4
  base_model: Qwen/Qwen3.5-9B
5
- pipeline_tag: text-generation
6
  tags:
7
- - WeVisCut
8
  - video-editing
9
- - long-form-video
10
  - agent
11
  - tool-use
12
- - speech
13
  - reinforcement-learning
14
  - qwen3.5
15
  ---
16
 
17
- # WeVisCut
18
 
19
- WeVisCut is a 9B tool-using policy for long-form video production from a narration script. It plans and edits through a registered runtime: selecting tools, using their outputs, inspecting intermediate results, and repairing failures. It is not a standalone text-to-video generator.
20
 
21
- **Base model:** `Qwen/Qwen3.5-9B`
22
- **Model:** [Hugging Face](https://huggingface.co/tencent/WeVisCut)
23
- **Code:** [GitHub](https://github.com/Tencent/WeVisCut)
24
- **Project page:** [WeVisCut](https://xd-mu.github.io/WeVisCut/)
25
 
26
- ## Training
27
 
28
- The policy uses shared parameters across three roles: **Orchestrator**, **Generator**, and **Critic**. The paper describes a three-stage post-training process:
29
-
30
- 1. Supervised fine-tuning (SFT) and runtime-verified rejection-sampled fine-tuning (RFT).
31
- 2. Guided on-policy distillation (Guided-OPD), with teacher interventions gradually reduced.
32
- 3. Role- and context-factored DAPO (RCF-DAPO), using execution feedback to optimize decisions.
33
-
34
- The paper constructs executable trajectories from 3,750 completed narration-driven video projects and reports 134,771 training examples. The source videos are described as spanning technology and digital products, business and finance, natural science, and social sciences and humanities. These figures describe the paper's data construction; they do not imply that the source videos or dataset are available with this checkpoint.
 
 
 
35
 
36
- ## Evaluation
37
 
38
- The paper evaluates on 100 held-out script-to-video tasks, with five independent runs in the shared-tool evaluation. It reports five metrics: Script Content Fidelity (SCF), Visual-Narration Semantic Match (VNSM), Timeline Plan Realization (TPR), Long-Horizon Consistency (LHC), and End-to-End Rendering Yield (ERY).
39
 
40
- In the common-harness comparison, the paper reports **2,377 generated tokens per successful output minute** for WeVisCut and **78,763** for Claude Opus 5. This is a policy-generation cost measure, not end-to-end runtime or hardware throughput. The paper's Overall Score is a descriptive mean of the five metrics, not an independently validated sixth metric; the reported results should not be interpreted as evidence of statistical equivalence.
41
 
42
- ## Intended use and limitations
43
 
44
- Use WeVisCut for research on narration-driven editing workflows in a compatible tool runtime. The policy depends on registered tools, available media assets, and the runtime's state and validation contract; the checkpoint alone does not retrieve assets or render a complete video. Results may not transfer to unregistered runtimes, other asset distributions, or visual styles outside the current template library. Visual-narration evaluation also relies on a vision-language model and may be affected by evaluator bias or service changes.
45
 
46
  ## Citation
47
 
48
- If you use this work, please cite the paper:
49
-
50
  ```bibtex
51
- @article{mu2026weviscut,
52
- title = {{WeVisCut}: The Whole Editing Room in One Agent --- From Narration to Long-Form Video},
53
- author = {Mu, Chenyu and Weng, Wenming and Wang, Qinrui and Ye, Xingsong and Yu, Hao and He, Jun and Sun, Chong and Li, Chen and Lyu, Jing and Yang, Erkun and Deng, Cheng},
54
- year = {2026}
 
 
 
55
  }
56
  ```
57
 
58
  ## License
59
 
60
- The model card metadata currently uses `license: other`. Check the model repository and the licenses of the base model, tools, and media assets before use; do not assume they share the same license.
 
2
  license: other
3
  library_name: transformers
4
  base_model: Qwen/Qwen3.5-9B
5
+ pipeline_tag: any-to-any
6
  tags:
 
7
  - video-editing
 
8
  - agent
9
  - tool-use
10
+ - long-horizon
11
  - reinforcement-learning
12
  - qwen3.5
13
  ---
14
 
15
+ # WeVisCut: A Long-Horizon Tool Agent for Edited Videos
16
 
17
+ **WeVisCut** is a 9B tool-using agent that turns knowledge articles or narration scripts into multi-minute edited videos. At each step, it reads the current editing state and selects one of 12 tools for planning, narration, asset retrieval, composition, inspection, or repair. It is not a standalone text-to-video model; use it with the [WeVisCut runtime](https://github.com/Tencent/WeVisCut), which executes the model's tool calls.
18
 
19
+ ## Deploy
 
 
 
20
 
21
+ Requirements: Python 3.11 or later and an NVIDIA GPU with about 24 GB of memory for bf16 inference. Install a PyTorch build compatible with your CUDA version before the GPU requirements below. The setup script installs Node.js, FFmpeg, and the renderer.
22
 
23
+ ```bash
24
+ git clone https://github.com/Tencent/WeVisCut.git
25
+ cd WeVisCut
26
+ pip install -r requirements.txt
27
+ pip install -r requirements-gpu.txt
28
+ bash scripts/setup_external_tools.sh
29
+ pip install -U "huggingface_hub[cli]"
30
+ hf download tencent/WeVisCut --local-dir checkpoints/weviscut-9b
31
+ bash run_studio.sh
32
+ ```
33
 
34
+ Open `http://<host>:8081/` to use the Studio. For UI-only mode without loading model weights, run `bash run_studio.sh --no-model`.
35
 
36
+ ## Model
37
 
38
+ The policy is based on Qwen3.5-9B and is trained on executable editing trajectories. Its runtime provides an editable timeline with pause/resume, manual and chat-based revisions, per-shot rerendering, and version rollback. Video quality depends on the configured retrieval, TTS, and rendering services and available templates.
39
 
40
+ ## Intended Use and Limitations
41
 
42
+ WeVisCut is intended for research and production assistance in long-form video editing. It does not guarantee factual accuracy or production quality; review generated narration and video before publication. The model weights require the WeVisCut tool environment to produce a video.
43
 
44
  ## Citation
45
 
 
 
46
  ```bibtex
47
+ @misc{weviscut2026,
48
+ title = {WeVisCut: A Long-Horizon Tool Agent for Turning Knowledge Articles into Edited Videos},
49
+ author = {WeVisCut Team},
50
+ organization = {Tencent},
51
+ year = {2026},
52
+ howpublished = {\url{https://xd-mu.github.io/WeVisCut/}},
53
+ note = {Technical report and code release}
54
  }
55
  ```
56
 
57
  ## License
58
 
59
+ The WeVisCut code is released under the [Apache License 2.0](https://github.com/Tencent/WeVisCut/blob/main/LICENSE). Model weights and third-party components are subject to their respective license terms.