Download docs/impl/res704.md from teawhite/maliang: direct link, hf CLI and curl.
- Browser
- Download file 38.7 kB
-
https://huggingface.co/teawhite/maliang/resolve/main/docs/impl/res704.md
- Command line
-
hf download hf://teawhite/maliang/docs/impl/res704.md
-
curl -L -o res704.md https://huggingface.co/teawhite/maliang/resolve/main/docs/impl/res704.md
res704: resolution-configurable gc25d stack (GC_RES), target 1280×704
Status 2026-10-23. Code done. Tests pass at both resolutions. Real-weight smokes ran at 1280×704 on GPUs 0–3: prep, verify,
ext_canvas, 4-GPU SFT, sample, three full-shape dry runs, stage1/2/3 memory probes and the eval twin. The 480p pipeline is unchanged:
the default GC_RES=832x480 reproduces every old constant and index table bit for bit (tested).
data/gc25d_v10 and outputs/sft_v10 were not touched.
Nothing has been launched at full scale. The commands for the full 704p prep → SFT → distill chain are at the end.
A review fix pass on the same day is summarised right below: deploy res guard, parse_res lower bound, launcher normalisation, chain
path guards, the capsule bbox, ext_canvas threads and verify thresholds. The final test and smoke numbers are there too.
Review fix pass (2026-10-23)
Ten reviewer findings were each verified first, then fixed, partly fixed or rejected. 832×480 output stays bit-identical (the old-constant
/ _IDX sha256 tests still pass; the capsule bbox default is unchanged at 832×480). Logs are in outputs/_smoke/res704_fix/.
| # | finding | verdict | what changed |
|---|---|---|---|
| 1 | deploy engines never check ckpt res |
confirmed, fixed | Res guard at the top of LiveGame.__init__ in the pending deploy patch (deploy_vocab/live_vocab.patch), before any weights are read; FastLive / gen_server inherit it. The same check is in live_path_check.py. ref/ untouched. Also fixed a regression the res704 change had introduced: live_path_check --no_inject (the documented deploy-contract check) crashed, because maliang's data_gc now reads GEO.LAT_H, which the deploy package's hard-coded geometry lacks. The twin side now imports maliang's geometry |
| 2 | parse_res accepts values that crash at import; odd CROP_W off-centre |
confirmed, fixed | parse_res requires H ≥ 384 (downscale < 2) and raises a GC_RES ValueError; 115 of the 450 previously accepted values were crashers. CROP_W is rounded to the nearest even number, so the crop is exactly centred and PLAYER_DST = frame centre at every legal res. 480p / 704p unchanged; 1152×640 goes 1267 → 1268. firstframe_flux copy updated. Tests cover all 335 accepted values (import-time asserts simulated) plus a subprocess import per listed res |
| 3 | launchers compare raw vs normalised GC_RES |
confirmed, fixed | train_sft.sh, common_gc.sh and chain_after_sft.sh normalise to geometry.RES_TAG after validating. GC_RES=1280X704 now passes against the 704 mini (probe S2: reaches the existing-ckpt gate; S4b: require_data_res OK) |
| 4 | chain path defaults ignore GC_RES; unanchored pgrep |
confirmed, fixed | At GC_RES ≠ 832x480 the chain requires SFT_OUT, D and DATA_ROOT. Before waiting it checks the res of DATA_ROOT, each existing $D/stage*/config.json and SFT=; it checks the picked SFT again. pgrep is anchored. All three runners now refuse another resolution's run dir before opening train.log (DG.check_res(run_dir=…)) |
| 5 | infer.load_ids raw-silhouette guard bypassed at 704p |
rejected (documented instead) | The 10/11 refusal is a gc25d_v8 (ellipse-marker) rule. For v10, silhouettes are the condition: collapse_parts(cond) = raw ids.png for an ids-vocab model, and docs/understand/infer_contract.md:285 already notes the rationale no longer holds. Enforcing it at 704p would block valid v10 ids-vocab inputs. The proposed "parts vocab without 12..21" check cannot apply because infer.py only accepts {0..6, 10, 11, 255}; v10 parts conditions go through sample_user.py. Documented in the module doc and load_ids |
| 6 / 8 | render_capsules pad is fixed in dst px, so it clips more at 704p |
confirmed, fixed differently | Scaling the pad only shrinks the error. At 704p, missed character px vs a pad-64 reference over 29 has_ext clips (every 3rd frame) are: pad 2 0.149 %, 2.93 0.103 %, 4 0.067 %, 6 0.030 %. The cause is the world-axis bbox, so cam_axes adds ± r along the camera right/up/forward axes. That gives 0.000 % at both resolutions; on at every res except 832×480, which keeps the old bbox bit for bit (open issue 2). (Finding 8's formula 2·max(1, H/480)·1.4 would give 2.8, not 2.0, at 480p.) The verify_prep 50 px gate is scaled by AREA_VS_480 |
| 7 | 704p answer rests on detail ratio alone | confirmed (evidence caveat) | scripts/res_test_edge.py adds SSIM / Laplacian correlation / edge F1 (table at the top). 704 vs 640 ties on SSIM and Laplacian correlation, wins on edge recall/F1, and loses 0.24 dB PSNR. Stated in "Why 1280×704" |
| 9 | ext_canvas thread env set after numpy import | confirmed, fixed | The four thread vars are forced to 1 at module top, before numpy (prep_v10 pattern). The plain $ENV (OMP=8) command is now safe |
| 10 | verify_prep 480p thresholds; fixed-shape docstrings | confirmed (thresholds), rejected (docstrings) | Visible-frame gate 50·AREA_VS_480; PSNR WARN lower bound 20 dB off 832×480 (21 at 832×480). The docstrings flagged in verify_prep.py / data_gc.py already carry an "these are the 832×480 example (1280×704: …)" qualifier, so they were left alone |
Smoke (1280×704, GPUs 0–3)
- ext_canvas, 704 mini, 85 clips, 64 workers, plain
$ENV(OMP_NUM_THREADS=8 inherited, overridden inside): ≈ 200 s, rc 0. Peak load average 78 while sharing the CPUs with a 28-process job, vs 1,850 before. unknown 0.169, self-agree f80 0.974, f76 0.969 (same as before). New vs old canvases (ext_canvas_old_vs_new.json): blocks / prompts identical in 85/85. 57,081 px changed in 467 of 2,380 frames, all of them character pixels the old bbox had clipped (+0.155 % character px, 0 removed). The old canvases are in.tmp/cond_ext_canvas_704mini_old/. - verify_prep, 704 mini, 8 spots: OK, 200 clips, 0 fails, 0 warns (was 2 warns: the two R10_s1512 clips at
20.26 / 20.42 dB are inside the [20, 40] band used off 832×480). The render stats use the camera-axis bbox: v10 player unoccluded IoU
median 0.9977 (p10 0.9968), v9_stylized player coverage 0.9997 → 1.0000; boss unchanged (
verify_mini.log). - 832×480 bit-identity of the bbox change:
render_capsulesdefault vs the pre-fix module on 550 real 480p frames (10 mini clips, every 2nd frame): 550/550 bitwise equal. - deploy path,
live_path_check(3 streamed cells vseval_rollout.deploy_rollout):- 480p EMA through the freshly patched package (
--no_inject,.tmp/deploy_patched_fix704): bitwise equal 4/4 cells (regression fixed); - 480p EMA through
ref/live.py (injected): bitwise equal 4/4; - 704p probe EMA through
ref/live.py atGC_RES=1280x704: frames 13×704×1280, bitwise equal 4/4, DiT ≈ 715 ms/cell (the same overhead-bound eager loop as at 480p, ≈ 700 ms); - refusals: the 704p EMA vs the patched package, or vs
ref/at default GC_RES, stops inlive_path_checkbefore loading. PatchedLiveGame(...)andFastLive(...)called directly raise in 0.00 s (deploy_guard_probes.log).
- 480p EMA through the freshly patched package (
- launcher probes (
launcher_probes.log): 12 cases (both mismatch directions, illegal res, upper-case res, chain without D, chain aimed at the 480p distill dir / 480p data / a 480p SFT ckpt, 480p default chain); all behave as intended. Nothing was launched. res_test_edge.py(GPU 0, ≈ 3 min): PSNR / detail reproduceres_test.jsonexactly for all 5 resolutions.
Tests (tests_default.log, tests_704.log)
- default
GC_RES,CUDA_VISIBLE_DEVICES=3: 160 passed, 4 skipped, 322 s (was 158 / 4; the +2 are the new tests below). This includestest_704_suite_subprocess. GC_RES=1280x704, CPU: 156 passed, 8 skipped, 188 s (was 154 / 8). The skips are the same kinds as before.- New or extended in
test_res.py:test_parse_res: rejects 640x352 / 608x320 / 32x96;test_parse_res_accepted_are_import_safe: all 335 accepted W×H;_IDXasserts hold, the crop is even and centred, aspect error ≤ 1 px;test_listed_res_import_subprocess: real import of geometry + firstframe_flux for 7 resolutions;PLAYER_DST= frame centre; an illegal res is aGC_RESValueError, never an AssertionError;test_guards:check_run_dir_res, plus stage 1/2/3main()refusing another resolution's run dir with onlyconfig.jsonleft in it;test_capsules_rays_consistent_across_res: with the camera-axis bbox and the default pad 2, both resolutions equal the pad-64 render on the synthetic oblique cameras; the 704p default uses it and the 480p default is the old bbox.
Why 1280×704 (and not 540p / 600-something p)
The user asked "540p or 600+p, whichever fits better". The lead's resolution test is scripts/res_test.py → outputs/char_diag/res_test.json.
It measures 40 player crops from 8 clips (5 frames each): VAE round trip at each resolution, upsampled back, compared with the native
1280×704 GT. It also times one 21-cell bidirectional DiT forward. scripts/res_test_edge.py → outputs/char_diag/res_test_edge.json
(added in review) reruns the same crops through the same code path and adds three metrics that check where the detail is, not just how
much of it there is: grey SSIM, the correlation between the Laplacians of reconstruction and GT, and Canny edge F1 with 1 px tolerance.
Its PSNR / detail values match res_test.json exactly.
| GC_RES | detail ratio | crop PSNR | SSIM | Laplacian corr | edge P / R / F1 | DiT forward (21 cells) | tokens / cell |
|---|---|---|---|---|---|---|---|
| 832x480 (current) | 0.375 | 24.74 | 0.634 | 0.207 | 0.933 / 0.624 / 0.733 | 0.262 s (1.00×) | 390 |
| 960x544 ("540p") | 0.419 | 25.24 | 0.655 | 0.242 | 0.941 / 0.652 / 0.759 | 0.360 s (1.37×) | 510 |
| 1024x576 | 0.437 | 25.47 | 0.665 | 0.255 | 0.943 / 0.664 / 0.768 | 0.409 s (1.56×) | 576 |
| 1152x640 ("600+p") | 0.468 | 25.93 | 0.686 | 0.288 | 0.946 / 0.686 / 0.784 | 0.546 s (2.08×) | 720 |
| 1280x704 (native) | 0.779 | 25.69 | 0.679 | 0.291 | 0.921 / 0.803 / 0.855 | 0.691 s (2.64×) | 880 |
Every non-native resolution goes through crop + INTER_AREA downscaling. That resampling throws away fine character detail before the VAE sees the frame. 480p → 640p improves every metric a little. Going from there to native 704p:
- Edge recall jumps from 0.69 to 0.80 (F1 0.78 → 0.86): many more of the GT's real edges come back.
- Placement metrics tie. SSIM is 0.686 vs 0.679 and Laplacian correlation 0.288 vs 0.291.
- Edge precision drops slightly (0.946 → 0.921), and crop PSNR is 0.24 dB lower than at 640p.
Caveat: the detail ratio (0.468 → 0.779) overstates the 640p → 704p gain. It measures Laplacian energy, so it also rewards the absent resampling low-pass, and part of the extra high frequency at 704p is VAE artifact rather than faithful detail. Lower edge precision and PSNR, and a flat SSIM, all show this. The honest summary:
- 704p beats 480p and 540p on every metric.
- Against 640p, 704p recovers clearly more of the real edges but is no better on SSIM or PSNR, and costs about 27 % more DiT compute (2.64× vs 2.08× of 480p).
- 704p is also Wan2.2-TI2V-5B's own resolution, and the data's. Frames and ids pass through unchanged (no crop or rescale), so 704p is still the pick for "600+p". 540p is a cheap middle ground that gains little.
All of these are one env var. If the real-time budget later rules out 704p, GC_RES=960x544 or 1152x640 runs through the same code. The
general rule is W, H multiples of 32, 384 ≤ H ≤ 704, W ≤ 1280 and aspect ≤ 1280:704. Each resolution needs its own prep / SFT, because a data
root and a ckpt are tied to one resolution.
The contract
code/gc25d/geometry.pyis the only source of resolution. It parsesGC_RES="<W>x<H>"once at import (default"832x480"). Rules: W and H are multiples of 32, 384 ≤ H ≤ 704, W ≤ 1280, and the aspect ratio is at most the source's 1280:704 (wider would need a vertical crop, which is rejected with a message). H ≥ 384 means the downscale factor is < 2. Below that, the centre-aligned_IDXtables cannot reach the first/last source row and column, so the import-time asserts would fail;parse_resnow rejects those values first with aGC_RESValueError. Derived values:DST_W, DST_H,RES_TAG = "<W>x<H>",MIN_H = 384;CROP_W = crop_w_for(W, H) = min(1280, 2·round(352·W/H)), i.e. 704·W/H rounded to the nearest even number, so the crop is exactly centred.CROP_X0 = (1280 − CROP_W)//2. 832×480 gives 1220 / 30, exactly as before; 1280×704 gives 1280 / 0. Only odd cases change, e.g. 1152×640 is now 1268 / 6 instead of 1267 / 6, which was 0.5 source px off-centre;SX, SY, SCALE,_IDX_X/_IDX_Y(centre-aligned nearest), andPLAYER_DST.PLAYER_DSTis the exact frame centre at every legal resolution (tested). It is computed multiply-then-divide, so 832×480 gives exactly (415.5, 239.5) as the old literal did, while plainsrc_to_dstwould give 239.49999999999997;- new:
LAT_H, LAT_W = H/16, W/16(30×52 → 44×80),POOL_H, POOL_W = H/4, W/4(120×208 → 176×320),TOK_PER_CELL(390 → 880),AREA_VS_480(for pixel-count thresholds),IDENTITY(1280×704: no crop, no scaling). resample_rgb/resample_idstake an identity fast path whenIDENTITYis set. It returns a C-contiguous copy and is tested to equal the general path, both same-size INTER_AREA and fancy indexing.- Child processes (accelerate ranks, DataLoader workers, prep's spawned ranks, ProcessPool workers) inherit the env var, so every process in a run agrees.
- Resolution guards. A missing
resalways means"832x480", so all existing 480p roots and ckpts keep working unchanged.- Data root:
index/prep_args.json["res"]. prep writes it, plusreson every index row. It is checked inGcLatentDatasetandGcRolloutDataset,sample.py,sample_user.py --export_clip,infer.py(clip modes),eval_rollout.py(also chain mode, which bypasses the dataset),ext_canvas.py,verify_prep.py, all three distill runners (inparse_args, before loading 5B weights) andscripts/train_sft.sh/run_stage*.sh(require_data_res). prep also refuses to write into an--outthat already holds another resolution (prep_args or rows), andmergerefuses mixed rows. - Ckpt: safetensors metadata
res.gc_dit.save_gcalways writes it, so SFT exports and every distill product (dits/ ema/ critic/) carry it, andtrain.export_metadataadds it explicitly.sample.py,sample_user.py,infer.py,eval_rollout.pyand the stage1/2/3 runners refuse a ckpt whose res ≠ GC_RES. The runners check the init, teacher and student ckpts, warm starts, and the dits/ema/critic picked up on resume.train.pydir-resume refuses atrainer_state.jsonof another res (resis now written there). A safetensors warm start with a different res only prints a note, because cross-resolution warm starting is legitimate: the DiT andgc_sem_condare shape-generic. - Distill run dir:
<OUT>/config.json["res"](missing = 832x480; no config.json = new dir). All three runners callDG.check_res(run_dir=…)→geometry.check_run_dir_resbefore openingtrain.log. A run aimed at another resolution's run dir therefore stops without appending to its logs and without loading 5B (tested: the refused dir still holds onlyconfig.json). - Deploy: the pending deploy patch
scripts/distill_smoke/deploy_vocab/live_vocab.patch, applied bymake_patched_deploy.sh, now adds a res check at the top ofLiveGame.__init__, before any weights are read. It refuses a ckpt whose metadata res (missing = 832x480) ≠ the engine'sG.DST_W×G.DST_H.fast_live.FastLiveandgen_serverinheritLiveGame, so they are covered too.scripts/distill_smoke/live_path_check.pydoes the same check against the geometry itslive.pyactually uses (maliang's when injecting, which follows GC_RES; the package's own 832×480 with--no_inject).ref/itself is still untouched, as with the vocab patch. - Every error message names
GC_RES, the value found, and the fix. Example:--ckpt …/step-8.safetensors 是在 1280x704 上训练 / 导出的(metadata res,缺失 = 832x480),本进程 GC_RES=832x480:设 GC_RES=1280x704,或换成 832x480 的 ckpt.
- Data root:
- Launchers. All three validate
GC_RESand normalise it togeometry.RES_TAG, so" 1280X704"becomes1280x704. Every shell string comparison and log line therefore uses the canonical value.scripts/train_sft.shexportsGC_RES(default 832x480), refuses to start if DATA's res ≠ GC_RES, and logs it in the banner.gcdistill/common_gc.shlogs[res] GC_RES=…and providesrequire_data_res, whichrun_stage{1,2,3}.shcall on real data; their banners printres=.scripts/chain_after_sft.shtakesGC_RES,D(distill root) andDATA_ROOT, and passes each stage itsOUT=$D/stage*.- With the default GC_RES its defaults equal the old 480p chain exactly; eval_rollout gets
--data_root. - With any other GC_RES it requires
SFT_OUT,DandDATA_ROOTexplicitly, because the defaults are 480p directories. - Before waiting for the SFT, it checks the res of DATA_ROOT, of every existing
$D/stage*/config.jsonand of an explicitSFT=. It checks the picked SFT ckpt again before stage 1. - The SFT wait loop's
pgrepis anchored like train_sft.sh's (--output $SFT_OUT( |$)), so a 480p chain no longer waits on a runningsft_v10_704trainer.
- With the default GC_RES its defaults equal the old 480p chain exactly; eval_rollout gets
What changed per file (shapes that were hard-coded → geometry)
| file | change |
|---|---|
geometry.py |
GC_RES parsing (H ≥ 384, even crop_w_for), derived constants, identity fast paths, root_res / check_root_res / ckpt_res / check_ckpt_res / check_run_dir_res, docstrings generalised |
dataset.py |
LATENT_SHAPE, IDS_SHAPE, mask grid from geometry; check_root_res in the constructor (ds.res) |
infer.py |
H, W, LAT_H, LAT_W, SRC_* from geometry; messages print the actual shapes; ckpt/data-root res checks; a source-resolution ids array is not resampled when source = target. The --ids 10/11 "raw silhouette" refusal (a gc25d_v8 ellipse-marker rule) can only tell raw from processed ids by shape, so at 1280×704 it does not apply. This is documented in the module doc and load_ids; v10 parts conditions go through sample_user.py |
sample.py, sample_user.py |
shape asserts / warmup latents from geometry; ckpt + data-root res checks; res in json; at 1280×704 a 1280×704 first frame / ids are taken as-is |
viz.py |
PANEL_H, PANEL_W = DST_H, DST_W; the title bar stays 48 px, total height H+48 (752 at 704p) is still a multiple of 16 (asserted) |
prep_v10.py |
LAT_SHAPE from geometry; res in prep_args.json (+ crop_x0/crop_w) and every row; refuses a foreign-res --out. At 1280×704: RGB = raw frame, ids = raw ids.png, label_parts at full res (identity _IDX) |
verify_prep.py |
res check (prep_args + every row), res in the report, log says px per frame (<res>). Thresholds set at 480p are scaled: the visible-frame gate is GT ≥ 50·AREA_VS_480 px; the PSNR WARN band is [21, 40] at 832×480 and [20, 40] elsewhere (busy clips sit ≈ 1 dB lower against native GT) |
ext_canvas.py |
data-root res check, res in the report; canvas cell stays 6 cm (world units). Forces OMP/OPENBLAS/MKL/NUMEXPR_NUM_THREADS=1 at the top, before numpy, like prep_v10. The old setdefault inside main() ran after numpy was imported and never overrode $ENV's 8 |
capsules.py |
render_capsules(..., cam_axes=None). The bbox also projects the endpoints ± r along the camera right/up/forward axes (columns of world_to_view). The default is on at every resolution except 832×480, which keeps the old world-axis bbox bit for bit (the existing data/gc25d_v10/cond_ext_canvas was drawn that way). Measured on 29 has_ext clips, every 3rd frame, vs a pad-64 reference: 704p clipped character px 0.149 % → 0; 480p 0.080 % (unchanged default) → 0 with cam_axes=True |
model.py |
docstrings only (already shape-generic) |
procmap.py |
player_px0 ≥ 500·AREA_VS_480; the default out dir gets a _<res> suffix at non-default res (the 480p benchmark cases are not overwritten) |
firstframe_flux.py |
runs in the PCE env (no cv2 ⇒ cannot import geometry) ⇒ copy of parse_res / crop_w_for (tested equal to geometry); output = DST; generation grid gen_grid(W,H) = 16·(a,b)·t with a:b = W:H reduced, t chosen so the area is closest to 0.9 MP and the grid is at least the output size (832×480 → 1248×720 as before, 1280×704 → 1280×704); MIN_PX scaled by area; clip_frame uses the crop rule; clip_ids refuses a cond of another res |
gcdistill/gc_dit.py |
save_gc writes res |
gcdistill/data_gc.py |
LAT_HW, check_res(ckpts=…, root=…, run_dir=…) (SystemExit), GcRolloutDataset root check + latent shape from geometry |
gcdistill/ar_tf_gc.py, cd_gc.py, dmd_gc.py |
synthetic shapes (30,52) → DG.LAT_HW; res checks (parse_args + every real-weight load + run dir before the logs open); res in config / logs / stage1 trainer_state; new --dry_run_full_hw (tiny model, full-res latent + sem grids); dmd budget gate: the KV / scoring / per-grad-frame terms × TOK_SCALE = TOK_PER_CELL/390 (1.0 at 480p ⇒ identical constants), and dry runs skip the gate (it fired on the 5B constants with tiny models whenever a GPU was visible — the two pre-existing test_stage3 failures under CUDA_VISIBLE_DEVICES=0) |
gcdistill/eval_rollout.py |
ckpt + data-root res check (before loading), res in summary |
train.py |
res in export metadata, config.json, trainer_state; dir-resume refuses another res; warm-start note |
scripts/train_sft.sh, gcdistill/common_gc.sh, run_stage{1,2,3}.sh, scripts/chain_after_sft.sh |
see above |
configs/accelerate_fsdp_dmd_4gpu.yaml |
new: 2gpu yaml with num_processes 4. Needed for the 704p stage3 probe, because 2 GPUs fail the budget gate at 704p (126.1 > 126.0 GiB) |
scripts/distill_smoke/deploy_vocab/{live_vocab.patch,make_patched_deploy.sh}, scripts/distill_smoke/live_path_check.py |
deploy res guard (see the contract); make_patched_deploy.sh self-checks that the guard is in the patched live.py; live_path_check asserts the frame shape from live.G instead of a hard-coded 480×832 |
scripts/res_test_edge.py |
new: alignment metrics (SSIM, Laplacian correlation, edge F1) for the resolution decision, same crops and code path as res_test.py |
Not changed: code/arope/ (vendored; already shape-generic), the other analysis scripts in scripts/ (genbench_compose.py, char_*,
distill_smoke/* are 480p tools tied to existing 480p outputs), and ref/gc25d-live-demo (deploy package, see open issues; its
guard ships as part of the pending patch).
Tests
tests/test_res.py(new):- default res: every old geometry name equals the hard-coded old constants, plus
_IDX_X/_IDX_Yby sha256; the shapes derived in dataset / infer / viz / prep / data_gc / firstframe_flux / dmd budget equal the old literals;resample_*equal the old implementation; parse_resgood and bad values;- guards: root / ckpt / prep out /
DG.check_res/save_gcwrites res; - a tiny DiT
GcTrainModulestep on the real latent + sem grids of the current res; infer.sample/viz.SideBySideshapes;- the firstframe_flux copy matches geometry for 8 resolutions;
- cross-resolution capsules: a 704p subprocess renders capsules, labels parts and casts ground points; sampled with the 480p
_IDX, they equal the 480p results (render / label bitwise, ground points ≤ 1e-6); test_704_suite_subprocess: reruns test_res / dataset (synthetic) / train / gcdit (+ data_gc rollout dataset) / stage1 / stage2 / stage3 dry runs / prep / sample_user / firstframe_flux / procmap withGC_RES=1280x704, and the 704p-only checks (identity paths, derived constants, budget scaling) run inside it.
- default res: every old geometry name equals the hard-coded old constants, plus
- Existing tests made resolution-aware, but only where they encoded 480p legitimately: synthetic roots write
prep_args.json, rows carryres, shapes come from geometry, and "real mini" tests usedata/gc25d_v10_miniat 480p anddata/gc25d_v10_704_miniat 704p (skipped if the res differs). Files:test_dataset,test_gcdit,test_stage1,test_stage3,test_prep,test_sample_user,test_firstframe_flux. - Results of the implementation pass (
outputs/_smoke/res704/tests_default.log,tests_704.log; the review fix pass numbers are in the section at the top):- default
GC_RES,CUDA_VISIBLE_DEVICES=3: 158 passed, 4 skipped (326 s, includes the 704p subprocess suite). The skips are the two GPU/real-weight gates, which were skipped before too, and two 704p-only tests. Before this change the same command had 2 failures (the dmd budget gate in dry runs). GC_RES=1280x704, CPU, with the 704 mini READY: 154 passed, 8 skipped (189 s). The real-mini tests (test_dataset::test_real_mini,test_stage1::test_7,test_stage3::test_18) run againstdata/gc25d_v10_704_mini. The skips are the 4 default-only tests in test_res, the 2 GPU/real-weight gates, and 2 tests that need the 480pdata/gc25d_v10.
- default
Real smoke at 1280×704 (GPUs 0–3, all logs in outputs/_smoke/res704/)
Mini prep (prep_mini.log): runs R10_s1501, R10_s1512, P9C_autumn_s1941, P9_golden_s2909 → data/gc25d_v10_704_mini (200 clips, 2 GPUs × 12 workers).
- VAE: 2.15 s/clip (480p: 1.07). Throughput 0.45 clips/s per GPU (GPU-bound; 480p was 0.72, worker-bound). GPU peak 11.4 GiB (480p: 6.5).
- Worker: 6.2 s/clip (label 3.9 s, write 1.3 s) × 12 workers per GPU ⇒ not the bottleneck.
- Extrapolated full set: 12,800 clips on 8 GPUs ≈ 60 min (the 480p full prep took 37 min).
- Disk: latent 7.1 MB/clip (480p 3.15), cond 2.04 MB, cond_ext 0.71 MB ⇒ full set ≈ 125–130 GB (480p: 53 GB).
verify_prep (verify_mini.log): OK, 200 clips, 0 fails, 2 warns, 250 s.
- A: res 1280x704 everywhere. B: 200/200. C: 8 spot clips: official
vae.encodere-encode exact 1.00000; cell 0 bf16-exact; collapse mismatch 0; relabel from tar json 28/28. - Capsule re-render from meta: v10 player unoccluded IoU median 0.9977, coverage 1.0000; boss 0.9967. The 480p mini gives 0.9978 / 0.9967.
- VAE reconstruction PSNR vs native GT: mean 26.62 dB (min 20.26, max 38.14). The 2 warns are two R10_s1512 clips at 20.3 / 20.4 dB, just below the [21, 40] band, which was calibrated at 480p (the same clips at 480p: 21.25 / 22.39). Native-resolution GT has more high frequencies, so expect about 1 dB lower PSNR on busy clips. These are warnings, not failures.
ext_canvas (ext_canvas_mini.log): 85 ext_look_change clips, 64 workers, 180 s. unknown 0.169, self-agreement f80 0.974, f76 0.969.
480p full-set values: 0.157 / 0.977 / 0.969. About 80 s per clip with --report (480p ≈ 52 s).
The implementer's first attempt ran without OMP_NUM_THREADS=1, and each of the 64 fork-workers started a full BLAS thread pool (load
average 1,850). ext_canvas.py now forces single-threaded BLAS itself (review fix pass), so the plain $ENV command is safe. These
canvases were then re-rendered with the camera-axis bbox (see the review fix pass).
SFT smoke (outputs/_smoke/sft704, outputs/_smoke/sft704.launcher.out): 4 GPUs (0–3) ZeRO-2, micro 1 × GA 2, 8 steps,
val at step 8 (4 clips × 3 t × 3 variants), export + save_state.
480p (_smoke/sft_timing, same 4-GPU GA2 setup) |
704p | |
|---|---|---|
| s/step (steady) | 2.40 | 5.92 (2.47×), data wait 0.03 s |
| peak allocated / GPU | 35.2 GiB | 37.9 GiB (max over ranks 40.0) |
| first step | — | 13.0 s |
- Grad checkpointing keeps activations small, so memory barely moves. From 4 to 8 GPUs ZeRO-2 halves the optimizer/grad shard (480p: 35.2 → 24.4 GiB), so 8 GPUs at 704p ≈ 27–29 GiB/GPU.
sem_rmsrises from 0 (zero init), and val runs (gaps ~0 after 8 steps, as expected).step-8.safetensors: 831 BF16 keys,check_gc_ckptOK, metadatares=1280x704.trainer_state.jsonhasres. The state slot is 66 GB (4 GPUs).- Full run estimate (8 GPUs, GA2, 20,000 steps): per-GPU work is the same as at 4 GPUs, so ≈ 5.9–6.1 s/step ⇒ ≈ 33–34 h of training. Add validation (every 500 steps, ~30 s) and saves (every 1000 steps: 10 GB export + state slot of roughly 70 GB at 8 GPUs) ⇒ ≈ 35 h. At 480p the same run is ≈ 14 h.
sample.py (sample_step8.log): 1 val clip (02_Autumn_Woodland_Shrine__P9_golden_s2909__002400), 8 steps.
- Sampling 6.6 s (0.83 s/step, one forward per step). Peak 22.6 GB including the non-tiled VAE decode of 81×704×1280. Wall 52 s.
- Side-by-side mp4 is 752×3840, 81 frames. PSNR is meaningless after 8 SFT steps (15.9 dB; VAE ceiling 33.6).
- Refusals checked: the 704p ckpt at default GC_RES, and the 480p mini at GC_RES=1280x704. Both stop before loading, with the messages above.
train.py --resume outputs/_smoke/sft704/stateat default GC_RES stops with… 是 res=1280x704 的存档(缺失 = 832x480),本次 GC_RES=832x480.train_sft.shrefuses both mismatch directions and an illegalGC_RES(1280x640, aspect wider than the source).
Distill dry runs at 704p shapes (dry_s{1,2,3}.log): --dry_run --dry_run_full_hw. Tiny models, but the latent is 44×80 and sem
ids are 704×1280 (verified: input_latents (48,5,44,80), sem_ids (17,704,1280)). stage1 13 s, stage2 15 s, stage3 21 s, all exit 0.
Exports carry res=1280x704.
Real-weight probes, 704p (init = the SFT smoke ckpt; quality irrelevant):
| stage | config | result |
|---|---|---|
| 3 DMD | FSDP1, 2 GPUs | refused by the budget gate: predicted 126.1 GiB > 0.9 × 140 = 126.0 (stage3_probe_2gpu_gate_abort.log) |
| 3 DMD | FSDP1, 4 GPUs (0–3), 2 steps, PROFILE_MEM, N = 22 / 27 (stage3_probe/) |
gate predicted 84.1 GiB. Measured peak 68.5 GiB/GPU (step 0: after rollout 44.7, after critic 61.2; step 1 with N=27: rollout peak 68.2). Probe gnorm 12.1; step 0 dmd = gnorm_g = 0 exactly; replay_dev = 0; step 2: 33 s (rollout 5.9 s, exit 0); step 1: 77 s (first step, includes warm-up). dits/critic/ema exported with res |
| 1 TF | ZeRO-2, 2 GPUs (0,1), GA1, 3 steps (stage1_probe/) |
11.7 s/step (480p 2-GPU smoke: 3.5–3.7), peak 67.7 GiB (480p: 56.5). gnorm 2.9–4.2; export + state OK |
| 2 CD | ZeRO-2, 2 GPUs (2,3), GA1, 3 steps (stage2_probe/) |
17.2 s/step (480p: 5.1), peak 98.3 GiB (480p: 85.0). dits/ + ema/ exported |
The teacher-forcing double stream (2 × 21 × 880 = 36,960 tokens) makes attention the dominant cost at 704p, so stage1/2 slow down by 3.2–3.4×, more than the SFT's 2.5×. Going from 2 to 8 GPUs removes ≈ 30 GiB of ZeRO-2 shard per GPU (optimizer m/v/master + grads, (1/2 − 1/8) of ≈ 80 GB). Expected 8-GPU peaks: stage1 ≈ 38 GiB, stage2 ≈ 68 GiB. Both fit.
8-GPU stage3 at 704p: of the 68.5 GiB, ≈ 41.9 is FULL_SHARD state (167.7 GiB / 4). At 8 GPUs that state is 21.0, so expect ≈ 48 GiB/GPU peak (gate prediction 63.2 GiB). At 480p the expectation was ≈ 40 GB. It fits comfortably.
Wall-clock plan at 704p, 8 GPUs (per-GPU work at 8 GPUs equals the measured smokes, because the batch per GPU is the same):
| step | 704p | 480p (for reference) |
|---|---|---|
| prep / verify / ext_canvas | ≈ 1 h / ≈ 0.75 h / ≈ 0.6 h | 37 min / 27 min / — |
| SFT 20k steps (GA2) | ≈ 35 h (5.9 s/step + val/save) | ≈ 14 h (2.4–2.5 s/step) |
| stage1 TF 8000 steps | ≈ 26 h (11.7 s/step) | ≈ 8 h |
| stage2 CD 8000 steps | ≈ 38 h (17.2 s/step) | ≈ 11 h |
| stage3 DMD 500 steps | ≈ 5–8 h (33–77 s/step in the 4-GPU probe; rollout length varies) | ≈ 3 h |
| total | ≈ 4.5 days | ≈ 1.5 days |
Stage-2 length (8000) is the documented quality lever. If the calendar is tight, shorten stage 1 first, not stage 2.
Deploy twin (eval_rollout.py) at 704p (eval_clip28_wan/, eval_chain121_tae/, speed_{704,480}/): runs end to end with the
probe EMA. clip28 + Wan VAE: 55 s wall. chain121 + TAEHV: 95 s. Same GPU, idle, chain121 + TAEHV, DiT only:
704p 420 ms/cell vs 480p 361 ms/cell. This eval loop is launch-overhead-bound (5 single-cell forwards per cell, eager PyTorch). The
compute ratio is ≈ 2.6× (table above), so an optimised deploy path (fast_live.py-style) will pay close to the compute ratio. Real time
needs ≤ 250 ms per cell (4 frames at 16 fps). 704p real-time is not established here. It needs the deploy package (see open issues).
Commands (M=/opt/dlami/nvme/zhiyangdeng/maliang)
M=/opt/dlami/nvme/zhiyangdeng/maliang
ENV="DIFFSYNTH_SKIP_DOWNLOAD=True PYTHONPATH=$M/code/gc25d:$M/code/arope HF_HOME=/opt/dlami/nvme/zhiyangdeng/.cache/huggingface TMPDIR=$M/.tmp OMP_NUM_THREADS=8 GC_RES=1280x704"
# 1) full 704p prep, 8 GPUs (≈ 60 min, ≈ 130 GB; resumable: rerun the same command)
env $ENV CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 $M/.venv/bin/python $M/code/gc25d/prep_v10.py \
--out $M/data/gc25d_v10_704 --gpus 0,1,2,3,4,5,6,7 --workers_per_gpu 12 > $M/outputs/prep_full/prep_704.log 2>&1
# 2) verify (480p full verify took 27 min; expect ~45 min)
env $ENV CUDA_VISIBLE_DEVICES=0 $M/.venv/bin/python $M/code/gc25d/verify_prep.py \
--root $M/data/gc25d_v10_704 --spot 16 --device cuda:0 --workers 64 > $M/outputs/prep_full/verify_704.log 2>&1
# 3) stage-3 extension canvases (4,772 ext_look_change clips; ≈ 70 s/clip/worker ⇒ ~35 min at 160 workers).
# ext_canvas.py forces OMP/OPENBLAS/MKL/NUMEXPR threads = 1 itself (the 8 in $ENV is overridden).
env $ENV $M/.venv/bin/python $M/code/gc25d/ext_canvas.py \
--root $M/data/gc25d_v10_704 --workers 160 > $M/outputs/prep_full/ext_canvas_704.log 2>&1
touch $M/data/gc25d_v10_704/READY # after verify says OK (exit code 0, "OK" in verify_704.log)
# 4) full SFT from scratch, same recipe as 480p (8 GPUs × micro 1 × GA 2 = 16, 20k steps, lr 1e-5 / cond_lr 1e-4, warmup 500)
GC_RES=1280x704 DATA=data/gc25d_v10_704 OUT=outputs/sft_v10_704 NGPU=8 GA=2 \
setsid nohup bash $M/scripts/train_sft.sh > $M/outputs/sft_v10_704.launcher.out 2>&1 &
# crash resume: same + RESUME=outputs/sft_v10_704/state
# 5) distill chain at 704p (waits for sft_v10_704/done.json, picks the teacher, stage1 8000 → stage2 8000 → stage3 500 → eval).
# At GC_RES ≠ 832x480 the chain refuses to start unless SFT_OUT, D and DATA_ROOT are all given; it checks their res up front.
mkdir -p $M/outputs/distill/gc25d_v10_704
GC_RES=1280x704 SFT_OUT=$M/outputs/sft_v10_704 D=$M/outputs/distill/gc25d_v10_704 DATA_ROOT=$M/data/gc25d_v10_704 \
setsid nohup bash $M/scripts/chain_after_sft.sh > $M/outputs/distill/gc25d_v10_704/chain.log 2>&1 &
# to skip the wait/pick: add SFT=<abs path of step-N.safetensors>
# per stage by hand: GC_RES=1280x704 DATA_ROOT=$M/data/gc25d_v10_704 OUT=$M/outputs/distill/gc25d_v10_704/stage1_tf \
# SFT=<sft ckpt> bash $M/code/gc25d/gcdistill/run_stage1.sh (likewise stage2 AR=…, stage3 STUDENT=… SFT=…)
# sampling / eval at 704p
env $ENV CUDA_VISIBLE_DEVICES=0 $M/.venv/bin/python $M/code/gc25d/sample.py --ckpt $M/outputs/sft_v10_704/step-N.safetensors \
--data_root $M/data/gc25d_v10_704 --val_n 8 --out_dir $M/outputs/sft_v10_704/eval_step-N
# deploy-path parity at 704p: the live.py model path (ref/ live.py, maliang modules injected ⇒ follows GC_RES) vs
# eval_rollout.deploy_rollout, 3 streamed cells, must be bitwise equal
env $ENV AROPE_ROOT=$M/code/arope CUDA_VISIBLE_DEVICES=0 $M/.venv/bin/python $M/scripts/distill_smoke/live_path_check.py \
--ckpt $M/outputs/distill/gc25d_v10_704/stage3_dmd/ema/ema_step500.safetensors --data_root $M/data/gc25d_v10_704 \
--out $M/outputs/distill/gc25d_v10_704/eval/live_path
Open issues / caveats
- Deploy package is still 480p.
ref/gc25d-live-demo/code/gc25d/{geometry.py,codeworld/*}is a separate hard-coded 832×480 copy (e.g.gen_server.pyhasH, W = 480, 832) and was not touched. A 704p deploy needs it ported (geometry + code world + server shapes). Until then the pending deploy patch refuses a 704p ckpt inLiveGame.__init__, before any weights are read; it no longer loads silently at 480p. The malianggeometry.pywas deliberately not copied into the patched package: the package would then follow an inheritedGC_RES=1280x704, and the guard would pass a 704p ckpt into a server that still hard-codes 480×832. The model path itself is resolution-generic:live_path_checkatGC_RES=1280x704(injected maliang modules) streams 3 cells at 704×1280, bitwise equal toeval_rollout(see the review fix pass). capsules.render_capsulesbounding box at 832×480. The world-axis bbox (endpoints ± r along the world axes) underestimates a sphere's screen radius under an oblique camera. The default is nowcam_axeson (adds ± r along the camera axes) at every resolution except 832×480, which keeps the old bbox bit for bit, sodata/gc25d_v10/cond_ext_canvasand the 480p verify reports reproduce. At 480p the old bbox misses 0.080 % of character px (29 has_ext clips, every 3rd frame);cam_axes=Truemisses 0. Turning it on at 480p is a one-line default change inrender_capsules(the 480p code world and canvases would then gain those pixels); that is the lead's call.- Verify PSNR WARN band: [21, 40] at 832×480 (as before), [20, 40] at other resolutions (native GT is ≈ 1 dB lower on busy clips).
- Full-res Wan-VAE decode for long chains (
eval_rollout --vae wan --chain_cells 241) at 704p was not tried. The chain eval inchain_after_sft.shalready uses TAEHV for chain241 and Wan only for clip28 (which runs fine: 55 s). - Smoke artifacts take ≈ 200 GB under
outputs/_smoke/{sft704,res704}. Most of it is thestate_Aslots ofsft704andres704/stage1_probe(≈ 66 GB each). They can be deleted, butsft704/step-8.safetensorsis the 704p init the probes used. - SPEC §1/§5 text and the other impl docs describe the 832×480 instance. Their shapes are what geometry gives at the default
GC_RES.