Download HF_MIGRATION_HANDOFF.md from Tinghong-Ye22/code-HFD: direct link, hf CLI and curl.
- Browser
- Download file 21.1 kB
-
https://huggingface.co/Tinghong-Ye22/code-HFD/resolve/main/HF_MIGRATION_HANDOFF.md
- Command line
-
hf download hf://Tinghong-Ye22/code-HFD/HF_MIGRATION_HANDOFF.md
-
curl -L -o HF_MIGRATION_HANDOFF.md https://huggingface.co/Tinghong-Ye22/code-HFD/resolve/main/HF_MIGRATION_HANDOFF.md
Hugging Face 工作区迁移交接
完成时间:2026-10-16T03:41:06Z
Hugging Face 账户:Tinghong-Ye22
验收结果
- 最终审计:通过。
- 目标仓库:46 个;远端实际目标仓库:46 个。
- 公开状态:全部公开;缺失仓库 0;意外仓库 0。
- 可浏览代码层:40/40 个来源核验完成。
- 完整代码与脚本快照:40/40 个来源核验完成,共 185 个归档分片、3,075.172 GiB。
- 数据、环境与导出快照:23/23 个来源核验完成,共 126 个归档分片、2,221.278 GiB。
- 最终再次向 Hub 查询并核对 374 个归档分片或 manifest,其中 311 个归档分片逐个比较远端 SHA-256 与本地 manifest;大小或哈希不一致 0。
- 分别从公开仓库
code-qwen3-vl与聚合仓库videogen-hfd重新下载代码和数据 manifest/快照;2 项恢复测试的 SHA-256、zstd -t与实际tar解包均通过,验证本文两种恢复命令可用。 - 本地源目录未移动、未删除,迁移只读打包;临时归档在完成远端核验前一直保留。
仓库清单
数据来源映射
| 本地来源 | HF 仓库 | 分片数 | 压缩后大小 |
|---|---|---|---|
/mnt/data/cpfs/user_workspace/yetinghong/envs/gmr |
envs |
1 | 4.172 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/envs/gvhmr |
envs |
1 | 3.065 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/hfd_experiments |
hfd_experiments |
23 | 453.419 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/hfd_exports |
hfd_exports |
3 | 58.927 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/charades_ego_v1 |
videogen-charades_ego_v1 |
2 | 21.120 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_audio_command_continuous_office_v1 |
videogen-hfd |
1 | 4.426 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_backchannel_commands_v2_short |
videogen-hfd |
1 | 0.002 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_control_commands_v1 |
videogen-hfd |
1 | 0.002 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_gqa_8voice_v1 |
videogen-hfd |
1 | 1.364 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_gqa_v1 |
videogen-hfd |
1 | 4.297 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_hesitation_pause_v1 |
videogen-hfd |
1 | 3.609 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_hesitation_pause_v2 |
videogen-hfd |
1 | 0.029 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_long_video_candidates_v1 |
videogen-hfd |
4 | 66.042 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_swa_vqav2_v1 |
videogen-hfd |
2 | 37.363 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_video_vqa_50k_v1 |
videogen-hfd |
40 | 792.587 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_video_vqa_8voice_v1 |
videogen-hfd |
1 | 11.625 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_video_vqa_live_v1 |
videogen-hfd |
1 | 0.000 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_visual_dataset |
videogen-hfd |
32 | 635.535 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_visual_dataset_build_261001 |
videogen-hfd |
4 | 77.406 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_visual_dataset_recovered_261005 |
videogen-hfd |
1 | 6.885 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_visual_qa |
videogen-hfd |
2 | 27.812 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/hfd_vqav2_8voice_v1 |
videogen-hfd |
1 | 11.023 GiB |
/mnt/data/cpfs/user_workspace/yetinghong/videogen/interleaf |
videogen-interleaf |
1 | 0.568 GiB |
videogen-hfd 汇总所有 hfd_* 来源;envs 汇总 gmr 和 gvhmr。hfd_experiments、hfd_exports、videogen-charades_ego_v1 与 videogen-interleaf 分别使用独立仓库。请求中明确排除的 code 与 videogen 目录没有创建对应仓库。
明确排除的 code 目录:Auto-claude-code-research-in-sleep, EMDB_RICH_supplements, PromptHMR_reproduction, Psi0-v26-speed-profile-output, Psi0-yth-test, Psi0-yth-v24_test, S-NAMMD, SONIC-MJWarp-bwd, TRAM, Video-OnlineHMR, WHAM, Wan2.2, __MACOSX, agent, robot__arxiv_0901_filled。
明确排除的 videogen 目录:complex_motion_dataset、EMDB、RICH。
文件组织与恢复
代码仓库尽量提供两层内容:仓库根目录中的普通文件用于网页浏览;超大文件树受 Hub commit 限额影响时保留已经上传的普通文件,并以完整快照作为完整性依据。FULL_SNAPSHOT.tar.zst.part-* 是完整可恢复快照,FULL_SNAPSHOT.manifest.json 记录每个分片的字节数和 SHA-256。恢复某个代码或 script 仓库:
cat FULL_SNAPSHOT.tar.zst.part-* | zstd -d -c | tar -xf -
聚合数据仓库中,每个来源使用 <来源名>.tar.zst.part-* 和 <来源名>.manifest.json。恢复示例:
cat hfd_visual_dataset.tar.zst.part-* | zstd -d -c | tar -xf -
归档采用 POSIX tar、zstd level 1 和 20 GiB 二进制分片,保留权限、符号链接与空目录。每个 manifest 内含准确恢复命令和逐分片 SHA-256。
完整性边界
- 代码和脚本完整快照只排除
.git元数据;这是为了避免发布仓库本地 SSH 配置与密钥。项目工作文件、被 Git 忽略的文件、运行输出、缓存和用户授权公开的其他内容均包含在完整快照中。 - 数据、环境、实验与导出归档不使用内容过滤器,按整个来源目录打包。
- 网页可浏览层是便利副本,不作为完整性的唯一依据。完整性以归档分片、远端大小复核和 manifest SHA-256 为准。
- 可浏览层有 82412 个路径未在 Hub 普通文件列表显示;按仓库计数及每仓库前 20 个示例为:{"code-Movens-Zero_train": {"count": 26, "sample_first_20": ["scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/L/ckpt_25000__root40_v4/inferring.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/L/ckpt_25000__root40_v4/music/music_1912/ffmpeg.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/L/ckpt_25000__root40_v4/rendering.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/L/ckpt_25000__root40_v4/speech/speech_1983/ffmpeg.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/L/ckpt_25000__root40_v4/text/text_0997/ffmpeg.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/L/ckpt_25000__root40_v4/video/video_0800/ffmpeg.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/M/ckpt_26500__root40_v4/inferring.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/M/ckpt_26500__root40_v4/music/music_1912/ffmpeg.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/M/ckpt_26500__root40_v4/rendering.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/M/ckpt_26500__root40_v4/speech/speech_1983/ffmpeg.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/M/ckpt_26500__root40_v4/text/text_0997/ffmpeg.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/M/ckpt_26500__root40_v4/video/video_0800/ffmpeg.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/S/ckpt_26500__root40_v4/inferring.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/S/ckpt_26500__root40_v4/music/music_1912/ffmpeg.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/S/ckpt_26500__root40_v4/rendering.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/S/ckpt_26500__root40_v4/speech/speech_1983/ffmpeg.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/S/ckpt_26500__root40_v4/text/text_0997/ffmpeg.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224/previews/preview_20260927_v1/S/ckpt_26500__root40_v4/video/video_0800/ffmpeg.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224_a1e-4_o1e-5_nojerk/previews/preview_20260928_1500/S/ckpt_1500__root40_v4/music/music_1912/ffmpeg.log", "scale/cleanup_reports/20260928_201511/records/groups/sml_1m_224_a1e-4_o1e-5_nojerk/previews/preview_20260928_1500/S/ckpt_1500__root40_v4/speech/speech_1983/ffmpeg.log"]}, "code-Psi0-yth-v18": {"count": 6, "sample_first_20": ["test/infer_verify_2606160122/audio/movies__F-SNBR4jhY_0007/motion.npy", "test/infer_verify_2606160122/audio/movies__F-SNBR4jhY_0007/reference_motion.mp4", "test/infer_verify_2606160122/audio_command/movies_1mzl0Zh614A_0000/motion.npy", "test/infer_verify_2606160122/audio_command/movies_1mzl0Zh614A_0000/reference_motion.mp4", "test/infer_verify_2606160122/text/g1/csv_sonic/manual/text_prompt/motion.npy", "test/infer_verify_2606160122/text/g1/csv_sonic/manual/text_prompt/reference_motion.mp4"]}, "code-Psi0-yth-v24": {"count": 9, "sample_first_20": ["runtime_indexes/omni_hidden_states_v23_oss_mnt/audio_key_index.4c5d7bdd40af65dc4309.pkl", "runtime_indexes/omni_hidden_states_v23_oss_mnt/audio_key_index.80c03d89f0fc0a151be9.pkl", "runtime_indexes/omni_hidden_states_v23_oss_mnt/audio_key_index.90836b02463d0d36d477.pkl", "runtime_indexes/omni_hidden_states_v23_oss_mnt/video_manifest_prefix.3ffb7feafcd3e1977d63.pkl", "runtime_indexes/omni_hidden_states_v23_oss_mnt/video_manifest_prefix.91cd78750baaebafa630.pkl", "runtime_indexes/omni_hidden_states_v23_oss_mnt/video_manifest_prefix.df6f85b0697637dc45f5.pkl", "runtime_indexes/omni_hidden_states_v23_oss_mnt/video_split_inverse.1114d058e1ac5d85b006.npy", "runtime_indexes/omni_hidden_states_v23_oss_mnt/video_split_inverse.611877ef3be4a4547206.npy", "runtime_indexes/omni_hidden_states_v23_oss_mnt/video_split_inverse.acf938e62e2308b8330b.npy"]}, "code-GR00T-WholeBodyControl": {"count": 48961, "sample_first_20": [".github/workflows/docs.yml", "decoupled_wbc/init.py", "decoupled_wbc/control/init.py", "decoupled_wbc/control/base/init.py", "decoupled_wbc/control/base/env.py", "decoupled_wbc/control/base/humanoid_env.py", "decoupled_wbc/control/base/policy.py", "decoupled_wbc/control/base/sensor.py", "decoupled_wbc/control/envs/init.py", "decoupled_wbc/control/envs/g1/init.py", "decoupled_wbc/control/envs/g1/g1_body.py", "decoupled_wbc/control/envs/g1/g1_env.py", "decoupled_wbc/control/envs/g1/g1_hand.py", "decoupled_wbc/control/envs/g1/sim/init.py", "decoupled_wbc/control/envs/g1/sim/base_sim.py", "decoupled_wbc/control/envs/g1/sim/image_publish_utils.py", "decoupled_wbc/control/envs/g1/sim/metric_utils.py", "decoupled_wbc/control/envs/g1/sim/robocasa_sim.py", "decoupled_wbc/control/envs/g1/sim/sim_utilts.py", "decoupled_wbc/control/envs/g1/sim/simulator_factory.py"]}, "code-Psi0-eval": {"count": 17256, "sample_first_20": [".gitignore", "CHANGELOG.md", "README.md", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step1000000_latest/FROZEN_BUNDLE.json", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step1000000_latest/argv.txt", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step1000000_latest/dataset_statistics.json", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step1000000_latest/envs.txt", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step1000000_latest/run_config.json", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step656001/FROZEN_BUNDLE.json", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step656001/argv.txt", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step656001/dataset_statistics.json", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step656001/envs.txt", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step656001/run_config.json", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step996001_best/FROZEN_BUNDLE.json", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step996001_best/argv.txt", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step996001_best/dataset_statistics.json", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step996001_best/envs.txt", "artifacts/frozen_runs/posttrain_v1_trimodal_2608080815_step996001_best/run_config.json", "artifacts/frozen_runs/posttrain_v1_trimodal_2608191043_step1001001_best/FROZEN_BUNDLE.json", "artifacts/frozen_runs/posttrain_v1_trimodal_2608191043_step1001001_best/argv.txt"]}, "code-HFD_swa_stream_2609111628_hzoGRo": {"count": 16154, "sample_first_20": [".aris/meta/events.jsonl", ".aris/traces/experiment-audit/20260911_run01/001-initial-audit.meta.json", ".aris/traces/experiment-audit/20260911_run01/001-initial-audit.request.json", ".aris/traces/experiment-audit/20260911_run01/001-initial-audit.response.md", ".aris/traces/experiment-audit/20260911_run01/002-additional-artifacts.meta.json", ".aris/traces/experiment-audit/20260911_run01/002-additional-artifacts.request.json", ".aris/traces/experiment-audit/20260911_run01/002-additional-artifacts.response.md", ".aris/traces/experiment-audit/20260911_run01/003-runtime-artifacts.meta.json", ".aris/traces/experiment-audit/20260911_run01/003-runtime-artifacts.request.json", ".aris/traces/experiment-audit/20260911_run01/003-runtime-artifacts.response.md", ".aris/traces/experiment-audit/20260911_run01/004-launch-followup.meta.json", ".aris/traces/experiment-audit/20260911_run01/004-launch-followup.request.json", ".aris/traces/experiment-audit/20260911_run01/004-launch-followup.response.md", ".aris/traces/experiment-audit/20260911_run01/received_requests.md", ".aris/traces/experiment-audit/20260911_run01/reviewer_followup.md", ".aris/traces/experiment-audit/20260911_run01/reviewer_response.json", ".aris/traces/experiment-audit/20260911_run01/reviewer_response.md", ".aris/traces/experiment-audit/20260911_run01/run.meta.json", "after_training.py", "audit_questions.py"]}}。完整列表保存在本地状态文件,这些路径仍在对应完整快照中。
- 每个来源在其归档创建时形成独立快照;这不是所有目录同一时刻的全局原子快照。
- 归档期间检测到源文件并发变化的快照及准确路径为:{"code-Movens-Zero_train": {"consistency": "non-atomic: source files changed during tar read", "changed_paths": ["Movens-Zero_train/scale/experiments/fresh_mxl_20261009/split_xl/backups/XL/runs/fresh_XL_pureflow_cfg_8gpu.sg.flow1000.consta.lr1.0e-03.b512.gpus8.261009120940", "Movens-Zero_train/scale/experiments/fresh_mxl_20261009/split_xl/diagnostics/fresh_XL_step76500"]}}。其 tar 流与 zstd 分片均完整结束,逐分片 SHA-256 和远端 LFS SHA-256 仍会核对;限制在于这些活动路径代表归档读取到的版本,无法声明为暂停写入时刻的原子状态。
- 源目录中的 Hugging Face 登录令牌未写入本文档、Git 提交或远端清单。
审计依据
迁移状态与本地最终审计保存在 /mnt/data/cpfs/user_workspace/yetinghong/tmp/hf_migration_20261014/state/:
code_core_upload_state.jsoncode_archive_upload_state.jsonarchive_upload_state.jsonfinal_audit.jsonrestore_smoke.json
最终审计从原始目录重新推导目标集合,并与账户实际模型仓库集合严格比较;随后按 manifest 对全部远端归档路径再次查询大小,并逐分片比较 Hugging Face LFS SHA-256。只有全部检查通过,才生成本交接文件。