Upload folder using huggingface_hub
Browse files- README.md +47 -0
- model_ema_bf16.pt +3 -0
README.md
ADDED
|
@@ -0,0 +1,47 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: nvidia/Cosmos-Predict2.5-2B
|
| 4 |
+
tags:
|
| 5 |
+
- video-generation
|
| 6 |
+
- egocentric
|
| 7 |
+
- ablation
|
| 8 |
+
- cosmos-predict2
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# comind-actoractor-wo-sharedenv (iter 12200)
|
| 12 |
+
|
| 13 |
+
Checkpoint from the `comind_actoractor_wo_sharedenv` experiment, run as a guest run on a teammate's behalf
|
| 14 |
+
(experiment authored by a colleague; trained under a different account on borrowed GPU time).
|
| 15 |
+
|
| 16 |
+
## What this experiment is
|
| 17 |
+
|
| 18 |
+
Ablation of `comind_actoractor_personpose_shared_refpose_base2b` that removes the **shared environment context**
|
| 19 |
+
while keeping joint generation (V=2) and the two-person identity-coloured pose stream. Exactly two things change:
|
| 20 |
+
|
| 21 |
+
- **Own-context-only warping** -- `control_input_warped` / `control_input_visibility` are built from each view's own
|
| 22 |
+
past pool. Frames the parent sourced from the partner's pool become holes (black frame, visibility all-False).
|
| 23 |
+
Data: `ego_gen_comind_single`.
|
| 24 |
+
- **No reference frames** -- reference appearance, reference pose and posed-reference Plucker all off (R=0).
|
| 25 |
+
|
| 26 |
+
Consequence: the single-ego corpus stores cameras canonicalised to each view's own frame 0, so the shared
|
| 27 |
+
canonicalisation is a no-op and Plucker is per-view; the pair no longer shares a world frame. Cross-view
|
| 28 |
+
self-attention is the only remaining channel between the two egos.
|
| 29 |
+
|
| 30 |
+
## Training
|
| 31 |
+
|
| 32 |
+
- Warm start: base Cosmos 2B multiview (`nvidia/Cosmos-Predict2.5-2B`).
|
| 33 |
+
- 1x NVIDIA H200 (`fsdp_shard_size=1`, `grad_accum_iter=2`, global batch 2, matching the 2-GPU base2b arm's global batch).
|
| 34 |
+
- Trained from scratch for this arm (no resume) to **iter 12,200**; checkpoints every 200 iterations. Training was
|
| 35 |
+
stopped with SIGTERM shortly before the GPU allocation expired; this is the last saved checkpoint.
|
| 36 |
+
- Loss (rectified flow) is noisy, roughly 0.04-0.09 over the last iterations.
|
| 37 |
+
|
| 38 |
+
## Files
|
| 39 |
+
|
| 40 |
+
- `model_ema_bf16.pt` -- EMA weights in bfloat16 (recommended for inference), converted from the training-time
|
| 41 |
+
DCP/FSDP checkpoint via `scripts/convert_distcp_to_pt.py` in the cosmos-predict2.5 repo.
|
| 42 |
+
|
| 43 |
+
## Note
|
| 44 |
+
|
| 45 |
+
The training code lives in the colleague's own copy of the cosmos-predict2.5 tree
|
| 46 |
+
(`predict2_multiview/configs/vid2vid/experiment/nymeria_pose_2actor.py`, `comind_actoractor_wo_sharedenv`),
|
| 47 |
+
which was not modified for this run.
|
model_ema_bf16.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6af1d9e50a963ecad7d98aa77742888ff8c52e55fc94f5ae914430d1fee2263e
|
| 3 |
+
size 4119346643
|