Instructions to use XD-MU/ScriptAgent with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries
PEFT
How to use XD-MU/ScriptAgent with PEFT:
```
Task type is invalid.
```

How to use XD-MU/ScriptAgent with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="XD-MU/ScriptAgent")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("XD-MU/ScriptAgent")
model = AutoModelForMultimodalLM.from_pretrained("XD-MU/ScriptAgent", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use XD-MU/ScriptAgent with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "XD-MU/ScriptAgent"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XD-MU/ScriptAgent",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/XD-MU/ScriptAgent

SGLang

How to use XD-MU/ScriptAgent with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "XD-MU/ScriptAgent" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XD-MU/ScriptAgent",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "XD-MU/ScriptAgent" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XD-MU/ScriptAgent",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use XD-MU/ScriptAgent with Docker Model Runner:
```
docker model run hf.co/XD-MU/ScriptAgent
```

ScriptAgent / README.md

nielsr HF Staff

Update model card with paper, project, and code links

bf98a9d verified 6 months ago

3.1 kB

base_model: XD-MU/ScriptAgent
library_name: peft
pipeline_tag: text-generation
tags:
  - base_model:adapter:XD-MU/ScriptAgent
  - lora
  - transformers
arxiv: 2601.17737

ScriptAgent: Dialogue-to-Shooting-Script Generation Model

This model is a fine-tuned adapter (LoRA) designed to generate detailed shooting scripts from dialogue inputs. It is the implementation of ScripterAgent as described in the paper: The Script is All You Need: An Agentic Framework for Long-Horizon Dialogue-to-Cinematic Video Generation.

Project Page | Code | Demo

Model Description

ScriptAgent transforms conversational text (coarse dialogue) into structured, fine-grained, and executable cinematic scripts. It bridges the "semantic gap" between a creative idea and its cinematic execution, providing necessary context for video generation models, including character descriptions, scene settings, positions, and dialogue cues.

The model is compatible with ms-swift and supports efficient inference via the vLLM backend.

💡 Note: This repository contains a PEFT adapter (LoRA). To use it, you must merge it with the original base model or load it via ms-swift.

▶️ Inference with ms-swift (vLLM Backend)

To generate shooting scripts from dialogue inputs, use the following snippet with ms-swift. You can find DialoguePrompts here.

import os
from huggingface_hub import snapshot_download
from swift.llm import PtEngine, RequestConfig, InferRequest

os.environ['CUDA_VISIBLE_DEVICES'] = '0'

model_name = "XD-MU/ScriptAgent"
local_path = "./models/ScriptAgent"

# Download the model files
print("Downloading model...")
snapshot_download(
    repo_id=model_name,
    local_dir=local_path,
    local_dir_use_symlinks=False,
    resume_download=True
)

# Load using SWIFT
engine = PtEngine(local_path, max_batch_size=1)
request_config = RequestConfig(max_tokens=8192, temperature=0.7)

infer_request = InferRequest(messages=[
    {"role": "user", "content": "Your Dialogue Here"}
])
response = engine.infer([infer_request], request_config)[0]

print(response.choices[0].message.content)

Citation

If you find this work useful, please cite:

@article{directing2026,
  title={The Script is All You Need: An Agentic Framework for Long-Horizon Dialogue-to-Cinematic Video Generation},
  author={Mu, Chenyu and He, Xin and Yang, Qu and Chen, Wanshun and Yao, Jiadi and Liu, Huang and Yi, Zihao and Zhao, Bo and Chen, Xingyu and Ma, Ruotian and others},
  journal={arXiv preprint arXiv:2601.17737},
  year={2026}
}

Acknowledgments

Thanks to LLaMA-Factory for the SFT training framework.
Thanks to ms-swift for the GRPO training framework.