kimhosoo commited on
Commit
c45aff5
·
verified ·
1 Parent(s): 59ffed2

Upload folder using huggingface_hub

Browse files
Files changed (2) hide show
  1. README.md +47 -0
  2. model_ema_bf16.pt +3 -0
README.md ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: nvidia/Cosmos-Predict2.5-2B
4
+ tags:
5
+ - video-generation
6
+ - egocentric
7
+ - ablation
8
+ - cosmos-predict2
9
+ ---
10
+
11
+ # comind-actoractor-wo-sharedenv (iter 12200)
12
+
13
+ Checkpoint from the `comind_actoractor_wo_sharedenv` experiment, run as a guest run on a teammate's behalf
14
+ (experiment authored by a colleague; trained under a different account on borrowed GPU time).
15
+
16
+ ## What this experiment is
17
+
18
+ Ablation of `comind_actoractor_personpose_shared_refpose_base2b` that removes the **shared environment context**
19
+ while keeping joint generation (V=2) and the two-person identity-coloured pose stream. Exactly two things change:
20
+
21
+ - **Own-context-only warping** -- `control_input_warped` / `control_input_visibility` are built from each view's own
22
+ past pool. Frames the parent sourced from the partner's pool become holes (black frame, visibility all-False).
23
+ Data: `ego_gen_comind_single`.
24
+ - **No reference frames** -- reference appearance, reference pose and posed-reference Plucker all off (R=0).
25
+
26
+ Consequence: the single-ego corpus stores cameras canonicalised to each view's own frame 0, so the shared
27
+ canonicalisation is a no-op and Plucker is per-view; the pair no longer shares a world frame. Cross-view
28
+ self-attention is the only remaining channel between the two egos.
29
+
30
+ ## Training
31
+
32
+ - Warm start: base Cosmos 2B multiview (`nvidia/Cosmos-Predict2.5-2B`).
33
+ - 1x NVIDIA H200 (`fsdp_shard_size=1`, `grad_accum_iter=2`, global batch 2, matching the 2-GPU base2b arm's global batch).
34
+ - Trained from scratch for this arm (no resume) to **iter 12,200**; checkpoints every 200 iterations. Training was
35
+ stopped with SIGTERM shortly before the GPU allocation expired; this is the last saved checkpoint.
36
+ - Loss (rectified flow) is noisy, roughly 0.04-0.09 over the last iterations.
37
+
38
+ ## Files
39
+
40
+ - `model_ema_bf16.pt` -- EMA weights in bfloat16 (recommended for inference), converted from the training-time
41
+ DCP/FSDP checkpoint via `scripts/convert_distcp_to_pt.py` in the cosmos-predict2.5 repo.
42
+
43
+ ## Note
44
+
45
+ The training code lives in the colleague's own copy of the cosmos-predict2.5 tree
46
+ (`predict2_multiview/configs/vid2vid/experiment/nymeria_pose_2actor.py`, `comind_actoractor_wo_sharedenv`),
47
+ which was not modified for this run.
model_ema_bf16.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6af1d9e50a963ecad7d98aa77742888ff8c52e55fc94f5ae914430d1fee2263e
3
+ size 4119346643