Claw commited on
Commit
b4917e3
·
1 Parent(s): 1084b8c

Replace Comic-Gen with Chinese→Malay Video Dubber (CPU Basic)

Browse files
Files changed (4) hide show
  1. Dockerfile +4 -3
  2. README.md +16 -58
  3. app.py +220 -212
  4. requirements.txt +7 -14
Dockerfile CHANGED
@@ -1,9 +1,10 @@
1
  # HF-recommended base for Gradio Spaces
2
  FROM python:3.10-slim
3
 
4
- # Install system dependencies
5
  RUN apt-get update && apt-get install -y --no-install-recommends \
6
  git \
 
7
  && rm -rf /var/lib/apt/lists/*
8
 
9
  # Create user with UID 1000 (HF requirement)
@@ -14,7 +15,7 @@ ENV HOME=/home/user \
14
 
15
  WORKDIR $HOME/app
16
 
17
- # Copy requirements and install (single pip install, pin Gradio 4.44.1)
18
  COPY --chown=user requirements.txt .
19
  RUN pip install --no-cache-dir -r requirements.txt \
20
  "uvicorn>=0.14.0" \
@@ -27,4 +28,4 @@ ENV GRADIO_SERVER_NAME=0.0.0.0 \
27
  GRADIO_SERVER_PORT=7860 \
28
  PYTHONUNBUFFERED=1
29
 
30
- CMD ["python", "app.py"]
 
1
  # HF-recommended base for Gradio Spaces
2
  FROM python:3.10-slim
3
 
4
+ # Install system dependencies (ffmpeg for audio/video processing)
5
  RUN apt-get update && apt-get install -y --no-install-recommends \
6
  git \
7
+ ffmpeg \
8
  && rm -rf /var/lib/apt/lists/*
9
 
10
  # Create user with UID 1000 (HF requirement)
 
15
 
16
  WORKDIR $HOME/app
17
 
18
+ # Copy requirements and install
19
  COPY --chown=user requirements.txt .
20
  RUN pip install --no-cache-dir -r requirements.txt \
21
  "uvicorn>=0.14.0" \
 
28
  GRADIO_SERVER_PORT=7860 \
29
  PYTHONUNBUFFERED=1
30
 
31
+ CMD ["python", "app.py"]
README.md CHANGED
@@ -1,70 +1,28 @@
1
  ---
2
- title: Comic Scene Generator
3
- emoji: 🎨
4
- colorFrom: purple
5
- colorTo: pink
6
- sdk: docker
7
- sdk_version: "5.0.0"
8
  app_file: app.py
9
  pinned: false
10
- license: apache-2.0
11
- tags:
12
- - stable-diffusion
13
- - sdxl
14
- - noobai
15
- - manga
16
- - anime
17
- - cpu-basic
18
  ---
19
 
20
- # Comic Scene Generator
21
 
22
- 最小可用版:输入一句中文场景描述 → 自动扩写为适合 NoobAI-XL 的英文提示词 → 生成 640×960 漫画/二次元场景图。
23
 
24
- ## 功能特性
25
 
26
- - **Prompt 自动增强**:中文场景自动翻译 + 质量标签 + 风格标签(masterpiece, best quality, dynamic composition 等)
27
- - **NoobAI-XL 1.1 SDNQ UINT4**:量化模型 ~2.56 GB,Diffusers 仓库 ~3.63 GB,适配 16GB RAM
28
- - **CPU Basic 优化**:
29
- - `low_cpu_mem_usage=True` 全链路
30
- - VAE slicing + tiling
31
- - Attention slicing
32
- - `enable_model_cpu_offload()` 逐层卸载
33
- - 单张生成,关闭并行
34
- - **默认配置**:640×960、20 steps、Euler Ancestral、guidance 5.0
35
- - **输出**:PNG 格式
36
 
37
- ## 部署说明
 
 
 
38
 
39
- 1. 在 Hugging Face 创建 **Gradio Space**(SDK: Gradio, Hardware: CPU Basic)
40
- 2. 上传三个文件:`app.py`、`requirements.txt`、`README.md`
41
- 3. 等待 Build 完成(首次下载模型约 3.6 GB)
42
- 4. 打开网页,输入中文场景,点击 Generate
43
 
44
- ## 使用示例
45
 
46
- | 输入 | 风格 | 分辨率 | Steps |
47
- |------|------|--------|-------|
48
- | 一只愤怒的霸王龙站在暴雨中的史前森林里仰头怒吼,闪电照亮天空,低角度镜头,强烈的漫画构图,动态线条,丰富背景细节 | Manga / Anime | 640×960 | 20 |
49
- | 赛博朋克城市夜晚,霓虹灯招牌倒映在湿漉漉的街道上,孤独的少女撑着透明伞,电影感构图 | Manga / Anime | 640×960 | 20 |
50
- | 古风仙侠,白衣剑仙御剑飞行穿越云海,日出金光洒在云层上,宏大壮观,工笔画风 | Manga / Anime | 640×960 | 25 |
51
-
52
- ## 性能预期 (CPU Basic: 2 vCPU / 16GB RAM)
53
-
54
- | 分辨率 | Steps | 预估耗时 |
55
- |--------|-------|----------|
56
- | 640×960 | 20 | ~3-5 分钟 |
57
- | 640×960 | 30 | ~5-7 分钟 |
58
- | 512×768 | 20 | ~2-3 分钟 |
59
-
60
- > 首次冷启动需额外下载模型权重(~3.6 GB),之后缓存复用。
61
-
62
- ## 模型来源
63
-
64
- - **NoobAI-XL/NoobAI-XL-1.1-SDNQ-UINT4** (Apache-2.0)
65
- - SDNQ 量化:INT4 权重 + FP16 激活,体积约 1/3 原版
66
-
67
- ## 许可证
68
-
69
- Apache-2.0(遵循 NoobAI-XL 原模型许可)
70
- # Rebuild trigger Tue Sep 1 15:58:37 UTC 2026
 
1
  ---
2
+ title: Chinese To Malay Video Dubber CPU
3
+ emoji: 🇨🇳🇲🇾
4
+ colorFrom: blue
5
+ colorTo: green
6
+ sdk: gradio
 
7
  app_file: app.py
8
  pinned: false
 
 
 
 
 
 
 
 
9
  ---
10
 
11
+ # Chinese → Bahasa Melayu Video Dubber — CPU Basic
12
 
13
+ Designed for Hugging Face Spaces CPU Basic (2 vCPU / 16 GB RAM).
14
 
15
+ Pipeline:
16
 
17
+ Chinese MP4 → Whisper Small → Chinese → Malay translation → Meta MMS Malay TTS → FFmpeg → Malay MP4
 
 
 
 
 
 
 
 
 
18
 
19
+ Models:
20
+ - `openai/whisper-small`
21
+ - `Helsinki-NLP/opus-mt-zh-ms`
22
+ - `facebook/mms-tts-zlm`
23
 
24
+ This version does not clone the original speaker's voice. It is intentionally lightweight for CPU Basic.
 
 
 
25
 
26
+ Recommended test: 1–3 minute videos. Longer videos can take a long time on CPU.
27
 
28
+ FFmpeg/ffprobe are required by the Space runtime.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
app.py CHANGED
@@ -1,221 +1,229 @@
1
  import os
2
- import torch
3
- import gradio as gr
4
- from diffusers import StableDiffusionXLPipeline, AutoencoderKL
5
- from diffusers.schedulers import EulerAncestralDiscreteScheduler
6
- from transformers import CLIPTextModel, CLIPTextModelWithProjection, CLIPTokenizer
7
- from safetensors.torch import load_file
8
- import gc
9
-
10
- # ─── Config ──────────────────────────────────────────────────────────
11
- MODEL_ID = "Laxhar/noobai-XL-1.1" # HF repo (public, diffusers-compatible)
12
- DEVICE = "cpu"
13
- DTYPE = torch.bfloat16 # CPU 不支持 fp16 运算,改用 bfloat16(同样 2 字节/参数,省内存且 CPU 原生支持)
14
- DEFAULT_W, DEFAULT_H = 640, 960
15
- DEFAULT_STEPS = 20
16
- DEFAULT_GUIDANCE = 5.0
17
-
18
- # ─── Prompt enhancement (Chinese → English + quality tags) ───────────
19
- QUALITY_TAGS = (
20
- "masterpiece, best quality, ultra high res, "
21
- "dynamic composition, dramatic lighting, "
22
- "detailed background, manga style, anime style, "
23
- "sharp focus, vivid colors, professional illustration"
24
- )
25
-
26
- NEGATIVE_PROMPT = (
27
- "low quality, worst quality, bad anatomy, bad composition, "
28
- "blurry, pixelated, watermark, text, signature, logo, "
29
- "ugly, deformed, noise, artifacts, lowres, error, "
30
- "missing fingers, extra digits, fewer digits, cropped, "
31
- "jpeg artifacts, out of frame, mutation, mutated, "
32
- "extra limbs, malformed limbs, long neck, disfigured"
33
- )
34
-
35
- STYLE_PROMPTS = {
36
- "Manga / Anime": "manga style, anime style, screentone, ink lines, dynamic panel composition",
37
- "Western Comics": "western comic style, bold outlines, halftone dots, vibrant colors, dynamic action",
38
- "Manhwa": "manhwa style, webtoon style, clean lines, soft shading, vertical composition",
39
- "Sketch / Lineart": "sketch style, lineart, black and white, pencil drawing, detailed strokes",
40
- }
41
-
42
- def enhance_prompt(user_prompt: str, style: str) -> str:
43
- """中文场景 → 英文提示词 + 质量标签 + 风格标签"""
44
- # 简单的中文关键词映射(可扩展为更完整的翻译/扩写)
45
- cn_en_map = {
46
- "恐龙": "dinosaur",
47
- "霸王龙": "T-Rex",
48
- "暴雨": "heavy rain",
49
- "史前森林": "prehistoric forest",
50
- "仰头": "looking up",
51
- "怒吼": "roaring",
52
- "闪电": "lightning",
53
- "照亮": "illuminating",
54
- "天空": "sky",
55
- "低角度": "low angle",
56
- "镜头": "camera angle",
57
- "强烈": "intense",
58
- "漫画": "manga",
59
- "构图": "composition",
60
- "动态": "dynamic",
61
- "线条": "lines",
62
- "丰富": "rich",
63
- "背景": "background",
64
- "细节": "details",
65
- "愤怒": "angry",
66
- "站在": "standing in",
67
- "里": "in",
68
- "的": "",
69
- "一只": "a",
70
- "在": "in",
71
- "中": "in",
72
- "和": "and",
73
- "与": "and",
74
- }
75
- # 粗略翻译:把常见中文词替换成英文
76
- translated = user_prompt
77
- for cn, en in cn_en_map.items():
78
- translated = translated.replace(cn, en)
79
- # 如果全是中文,做简单拼接;如果已经有英文,保留
80
- if any('\u4e00' <= c <= '\u9fff' for c in translated):
81
- # 还含有中文,简单处理
82
- prompt_en = translated
83
- else:
84
- prompt_en = translated
85
- style_tag = STYLE_PROMPTS.get(style, STYLE_PROMPTS["Manga / Anime"])
86
- return f"{prompt_en}, {style_tag}, {QUALITY_TAGS}"
87
-
88
- # ─── Load pipeline (lazy, cached) ────────────────────────────────────
89
- _pipe = None
90
-
91
- def get_pipeline():
92
- global _pipe
93
- if _pipe is not None:
94
- return _pipe
95
-
96
- print("Loading NoobAI-XL SDNQ pipeline on CPU (bf16)...")
97
-
98
- # 1) VAE (单独加载,启用 slicing)
99
- vae = AutoencoderKL.from_pretrained(
100
- MODEL_ID,
101
- subfolder="vae",
102
- torch_dtype=DTYPE,
103
- low_cpu_mem_usage=True,
104
- )
105
- vae.enable_slicing()
106
- vae.enable_tiling()
107
-
108
- # 2) Text encoders & tokenizers (SDXL 双编码器)
109
- tokenizer_1 = CLIPTokenizer.from_pretrained(MODEL_ID, subfolder="tokenizer")
110
- tokenizer_2 = CLIPTokenizer.from_pretrained(MODEL_ID, subfolder="tokenizer_2")
111
- text_encoder_1 = CLIPTextModel.from_pretrained(
112
- MODEL_ID, subfolder="text_encoder", torch_dtype=DTYPE, low_cpu_mem_usage=True
113
- )
114
- text_encoder_2 = CLIPTextModelWithProjection.from_pretrained(
115
- MODEL_ID, subfolder="text_encoder_2", torch_dtype=DTYPE, low_cpu_mem_usage=True
116
- )
117
-
118
- # 3) Pipeline
119
- pipe = StableDiffusionXLPipeline.from_pretrained(
120
- MODEL_ID,
121
- vae=vae,
122
- text_encoder=text_encoder_1,
123
- text_encoder_2=text_encoder_2,
124
- tokenizer=tokenizer_1,
125
- tokenizer_2=tokenizer_2,
126
- scheduler=EulerAncestralDiscreteScheduler.from_pretrained(MODEL_ID, subfolder="scheduler"),
127
- torch_dtype=DTYPE,
128
- low_cpu_mem_usage=True,
129
- )
130
-
131
- # CPU 优化
132
- # 注意:enable_model_cpu_offload() 依赖 CUDA(会把模块按需搬到 GPU),
133
- # 纯 CPU 环境不能用,否则会报 "Found no NVIDIA driver"
134
- pipe.enable_vae_slicing()
135
- pipe.enable_vae_tiling()
136
- pipe.enable_attention_slicing("auto") # 注意力切片
137
- # 关闭 xformers / flash attention(CPU 不支持)
138
- try:
139
- pipe.disable_xformers_memory_efficient_attention()
140
- except Exception:
141
- pass
142
 
143
- pipe.to(DEVICE)
144
- _pipe = pipe
145
- print("Pipeline ready.")
146
- return pipe
147
-
148
- # ─── Generation ──────────────────────────────────────────────────────
149
- def generate(prompt, style, width, height, steps, guidance, progress=gr.Progress()):
150
- if not prompt.strip():
151
- return None, "请输入场景描述"
152
 
153
- progress(0.1, desc="增强提示词...")
154
- full_prompt = enhance_prompt(prompt, style)
 
155
 
156
- progress(0.2, desc="加载模型...")
157
- pipe = get_pipeline()
158
 
159
- progress(0.4, desc="生成中(CPU 慢,请耐心等待)...")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
160
  with torch.inference_mode():
161
- image = pipe(
162
- prompt=full_prompt,
163
- negative_prompt=NEGATIVE_PROMPT,
164
- width=width,
165
- height=height,
166
- num_inference_steps=steps,
167
- guidance_scale=guidance,
168
- num_images_per_prompt=1,
169
- ).images[0]
170
-
171
- progress(0.9, desc="保存 PNG...")
172
- # 保存到临时文件供 Gradio 返回
173
- out_path = "/tmp/comic_scene.png"
174
- image.save(out_path, "PNG")
175
- progress(1.0, desc="完成")
176
- gc.collect()
177
- torch.cuda.empty_cache() if torch.cuda.is_available() else None
178
- return out_path, f"✅ 生成完成\nPrompt: {full_prompt[:200]}..."
179
-
180
- # ─── Gradio UI ───────────────────────────────────────────────────────
181
- css = """
182
- #title { text-align: center; font-size: 1.5rem; font-weight: 700; margin-bottom: 0.5rem; }
183
- #subtitle { text-align: center; color: #666; margin-bottom: 1rem; }
184
- .generate-btn { background: linear-gradient(90deg, #667eea 0%, #764ba2 100%) !important; }
185
- """
186
-
187
- with gr.Blocks(css=css, title="Comic Scene Generator") as demo:
188
- gr.HTML('<div id="title">🎨 Comic Scene Generator</div>')
189
- gr.HTML('<div id="subtitle">输入一句中文场景 → 自动扩写 → NoobAI-XL 生成漫画场景图</div>')
190
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
191
  with gr.Row():
192
- with gr.Column(scale=1):
193
- prompt = gr.Textbox(
194
- label="Scene",
195
- placeholder="例:一只愤怒的霸王龙站在暴雨中的史前森林里仰头怒吼,闪电照亮天空,低角度镜头,强烈的漫画构图,动态线条,丰富背景细节",
196
- lines=4,
197
- )
198
- style = gr.Dropdown(
199
- label="Style",
200
- choices=list(STYLE_PROMPTS.keys()),
201
- value="Manga / Anime",
202
- )
203
- with gr.Row():
204
- width = gr.Slider(label="Width", minimum=320, maximum=1024, value=DEFAULT_W, step=64)
205
- height = gr.Slider(label="Height", minimum=320, maximum=1536, value=DEFAULT_H, step=64)
206
- steps = gr.Slider(label="Steps", minimum=10, maximum=50, value=DEFAULT_STEPS, step=1)
207
- guidance = gr.Slider(label="Guidance Scale", minimum=1.0, maximum=15.0, value=DEFAULT_GUIDANCE, step=0.5)
208
- btn = gr.Button("Generate", elem_classes="generate-btn", variant="primary")
209
-
210
- with gr.Column(scale=1):
211
- output_img = gr.Image(label="生成的漫画场景", type="filepath", interactive=False)
212
- status = gr.Textbox(label="Status", interactive=False)
213
-
214
- btn.click(
215
- generate,
216
- inputs=[prompt, style, width, height, steps, guidance],
217
- outputs=[output_img, status],
218
- )
219
 
220
  if __name__ == "__main__":
221
- demo.launch(server_name="0.0.0.0", server_port=7860, show_error=True)
 
1
  import os
2
+ import re
3
+ import subprocess
4
+ import tempfile
5
+ from pathlib import Path
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
 
7
+ import gradio as gr
8
+ import torch
9
+ import torchaudio
10
+ from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
11
+ from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
12
+ from transformers import VitsModel
 
 
 
13
 
14
+ ASR_MODEL = os.getenv("ASR_MODEL", "openai/whisper-small")
15
+ TRANSLATION_MODEL = os.getenv("TRANSLATION_MODEL", "Helsinki-NLP/opus-mt-zh-ms")
16
+ TTS_MODEL = os.getenv("TTS_MODEL", "facebook/mms-tts-zlm")
17
 
18
+ torch.set_num_threads(2)
19
+ DEVICE = "cpu"
20
 
21
+ _asr_processor = _asr_model = None
22
+ _tr_tokenizer = _tr_model = None
23
+ _tts_tokenizer = _tts_model = None
24
+
25
+ def run(cmd):
26
+ p = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
27
+ if p.returncode:
28
+ raise RuntimeError(p.stderr[-5000:])
29
+ return p.stdout
30
+
31
+ def video_duration(path):
32
+ return float(run(["ffprobe","-v","error","-show_entries","format=duration",
33
+ "-of","default=noprint_wrappers=1:nokey=1",str(path)]).strip())
34
+
35
+ def extract_audio(video, wav):
36
+ run(["ffmpeg","-y","-i",str(video),"-vn","-ac","1","-ar","16000",str(wav)])
37
+
38
+ def load_asr():
39
+ global _asr_processor, _asr_model
40
+ if _asr_model is None:
41
+ _asr_processor = AutoProcessor.from_pretrained(ASR_MODEL)
42
+ _asr_model = AutoModelForSpeechSeq2Seq.from_pretrained(
43
+ ASR_MODEL, low_cpu_mem_usage=True
44
+ ).to(DEVICE).eval()
45
+ return _asr_processor, _asr_model
46
+
47
+ def transcribe(wav):
48
+ processor, model = load_asr()
49
+ audio, sr = torchaudio.load(str(wav))
50
+ audio = audio.mean(0)
51
+ if sr != 16000:
52
+ audio = torchaudio.functional.resample(audio, sr, 16000)
53
+
54
+ total = audio.numel() / 16000
55
+ pieces = []
56
+ chunk = 30
57
+
58
+ for start in range(0, int(total + 0.999), chunk):
59
+ x = audio[start*16000:min(int(total*16000),(start+chunk)*16000)].numpy()
60
+ if len(x) < 400:
61
+ continue
62
+ inputs = processor(x, sampling_rate=16000, return_tensors="pt")
63
+ with torch.inference_mode():
64
+ ids = model.generate(
65
+ inputs.input_features,
66
+ language="zh",
67
+ task="transcribe",
68
+ max_new_tokens=256
69
+ )
70
+ text = processor.batch_decode(ids, skip_special_tokens=True)[0].strip()
71
+ if text:
72
+ pieces.append(text)
73
+ return " ".join(pieces)
74
+
75
+ def load_translation():
76
+ global _tr_tokenizer, _tr_model
77
+ if _tr_model is None:
78
+ _tr_tokenizer = AutoTokenizer.from_pretrained(TRANSLATION_MODEL)
79
+ _tr_model = AutoModelForSeq2SeqLM.from_pretrained(
80
+ TRANSLATION_MODEL, low_cpu_mem_usage=True
81
+ ).to(DEVICE).eval()
82
+ return _tr_tokenizer, _tr_model
83
+
84
+ def translate(text):
85
+ tokenizer, model = load_translation()
86
+ parts = [x.strip() for x in re.split(r'(?<=[。!?!?])\s*', text) if x.strip()]
87
+ if not parts:
88
+ parts = [text]
89
+
90
+ result = []
91
+ for part in parts:
92
+ inputs = tokenizer(part, return_tensors="pt", truncation=True, max_length=256)
93
+ with torch.inference_mode():
94
+ ids = model.generate(**inputs, max_new_tokens=256, num_beams=4,
95
+ renormalize_logits=True)
96
+ result.append(tokenizer.decode(ids[0], skip_special_tokens=True).strip())
97
+ return " ".join(result)
98
+
99
+ def load_tts():
100
+ global _tts_tokenizer, _tts_model
101
+ if _tts_model is None:
102
+ _tts_tokenizer = AutoTokenizer.from_pretrained(TTS_MODEL)
103
+ _tts_model = VitsModel.from_pretrained(
104
+ TTS_MODEL, low_cpu_mem_usage=True
105
+ ).to(DEVICE).eval()
106
+ return _tts_tokenizer, _tts_model
107
+
108
+ def tts(text, wav):
109
+ tokenizer, model = load_tts()
110
+ inputs = tokenizer(text, return_tensors="pt")
111
  with torch.inference_mode():
112
+ audio = model(**inputs).waveform.cpu()
113
+ torchaudio.save(str(wav), audio, model.config.sampling_rate)
114
+
115
+ def fit_audio(src, dst, target):
116
+ target = max(float(target), 0.1)
117
+ ratio = video_duration(src) / target
118
+ filters = []
119
+ while ratio > 2:
120
+ filters.append("atempo=2.0")
121
+ ratio /= 2
122
+ while ratio < 0.5:
123
+ filters.append("atempo=0.5")
124
+ ratio /= 0.5
125
+ filters.append(f"atempo={ratio:.6f}")
126
+ run(["ffmpeg","-y","-i",str(src),"-filter:a",",".join(filters),
127
+ "-t",f"{target:.3f}","-ar","24000","-ac","1",str(dst)])
128
+
129
+ def make_dub(ms, total, work):
130
+ parts = [x.strip() for x in re.split(r'(?<=[。!?!?])\s*|(?<=[,,])\s+', ms) if x.strip()]
131
+ if not parts:
132
+ parts = [ms]
133
+ weights = [max(1,len(re.sub(r'\s+','',x))) for x in parts]
134
+ total_w = sum(weights)
135
+
136
+ silent = work/"silent.wav"
137
+ run(["ffmpeg","-y","-f","lavfi","-i","anullsrc=r=24000:cl=mono",
138
+ "-t",f"{total:.3f}","-ar","24000","-ac","1",str(silent)])
139
+
140
+ inputs = ["-i",str(silent)]
141
+ labels = ["[0:a]"]
142
+ start = 0.0
143
+
144
+ for i, part in enumerate(parts, 1):
145
+ slot = total * weights[i-1] / total_w
146
+ raw, fitted = work/f"tts_{i}.wav", work/f"fit_{i}.wav"
147
+ tts(part, raw)
148
+ fit_audio(raw, fitted, slot)
149
+ inputs += ["-i",str(fitted)]
150
+ delay = int(start*1000)
151
+ labels.append(f"[{i}:a]adelay={delay}|{delay}[d{i}]")
152
+ start += slot
153
+
154
+ mix = "[0:a]" + "".join(f"[d{i}]" for i in range(1,len(parts)+1))
155
+ filt = ";".join(labels) + f";{mix}amix=inputs={len(parts)+1}:duration=first[a]"
156
+ out = work/"dub.wav"
157
+ run(["ffmpeg","-y",*inputs,"-filter_complex",filt,"-map","[a]",
158
+ "-ar","24000","-ac","1",str(out)])
159
+ return out
160
+
161
+ def process(video, keep_original, original_volume, progress=gr.Progress()):
162
+ if not video:
163
+ raise gr.Error("请上传中文视频。")
164
+ work = Path(tempfile.mkdtemp(prefix="zhms_cpu_"))
165
+ try:
166
+ src = Path(video)
167
+ audio = work/"source.wav"
168
+ progress(0.05, desc="提取声音…")
169
+ extract_audio(src, audio)
170
+ total = video_duration(src)
171
+
172
+ progress(0.15, desc="中文语音识别(CPU)…")
173
+ zh = transcribe(audio)
174
+ if not zh:
175
+ raise RuntimeError("没有识别到中文语音。")
176
+
177
+ progress(0.42, desc="翻译成 Bahasa Melayu…")
178
+ ms = translate(zh)
179
+
180
+ progress(0.58, desc="生成马来文配音…")
181
+ dub = make_dub(ms, total, work)
182
+
183
+ out = work/"malay_dub.mp4"
184
+ progress(0.9, desc="合成 MP4…")
185
+ if keep_original:
186
+ filt = f"[0:a]volume={float(original_volume):.3f}[o];[o][1:a]amix=inputs=2:duration=first[a]"
187
+ run(["ffmpeg","-y","-i",str(src),"-i",str(dub),"-filter_complex",filt,
188
+ "-map","0:v:0","-map","[a]","-c:v","copy","-c:a","aac","-b:a","160k",
189
+ "-shortest",str(out)])
190
+ else:
191
+ run(["ffmpeg","-y","-i",str(src),"-i",str(dub),
192
+ "-map","0:v:0","-map","1:a:0","-c:v","copy","-c:a","aac",
193
+ "-b:a","160k","-shortest",str(out)])
194
+
195
+ txt = work/"translation.txt"
196
+ txt.write_text("=== 中文识别 ===\n"+zh+"\n\n=== Bahasa Melayu ===\n"+ms,encoding="utf-8")
197
+ progress(1, desc="完成")
198
+ return str(out), str(txt), zh, ms
199
+ except Exception as e:
200
+ raise gr.Error(str(e))
201
+
202
+ with gr.Blocks(title="中文 → Bahasa Melayu(CPU Basic)") as demo:
203
+ gr.Markdown("""# 🇨🇳 → 🇲🇾 中文视频 → Bahasa Melayu
204
+ **Hugging Face CPU Basic 专用版本**
205
+
206
+ 上传中文视频 → 中文语音识别 → 马来文翻译 → 马来文 TTS → 输出 MP4。
207
+
208
+ - 不需要 GPU
209
+ - 不需要 API Key
210
+ - 不使用声音克隆
211
+ - 使用专门的 Chinese → Malay 翻译模型
212
+ - 使用 Meta MMS Malay TTS
213
+
214
+ ⚠️ CPU Basic 只有 2 vCPU / 16 GB RAM,建议先测试 1–3 分钟视频。
215
+ """)
216
+ video = gr.Video(label="上传中文视频", sources=["upload"], type="filepath")
217
  with gr.Row():
218
+ keep = gr.Checkbox(label="保留原声音作为背景", value=False)
219
+ volume = gr.Slider(0,0.3,value=0.06,step=0.01,label="原声音量")
220
+ button = gr.Button("🚀 开始翻译成马来文", variant="primary")
221
+ output = gr.Video(label="🇲🇾 马来文视频")
222
+ text_file = gr.File(label="识别 + 翻译文本")
223
+ with gr.Row():
224
+ zh_box = gr.Textbox(label="中文识别结果", lines=8)
225
+ ms_box = gr.Textbox(label="Bahasa Melayu", lines=8)
226
+ button.click(process,[video,keep,volume],[output,text_file,zh_box,ms_box])
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
227
 
228
  if __name__ == "__main__":
229
+ demo.queue(max_size=2).launch()
requirements.txt CHANGED
@@ -1,14 +1,7 @@
1
- # Core ML dependencies
2
- torch==2.4.0
3
- torchvision==0.19.0
4
- torchaudio==2.4.0
5
-
6
- diffusers==0.31.0
7
- transformers==4.44.0
8
- accelerate==0.33.0
9
- safetensors==0.4.4
10
- pillow==10.4.0
11
-
12
- # Gradio (use latest 5.x with latest gradio_client)
13
- gradio>=5.0.0
14
- gradio_client>=1.0.0
 
1
+ gradio>=5,<7
2
+ torch>=2.2
3
+ torchaudio>=2.2
4
+ transformers>=4.48,<5
5
+ sentencepiece>=0.2
6
+ safetensors>=0.4
7
+ numpy<2.3