Instructions to use AlterraLaniakea/jepa-guided-diffusion with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Cosmos
How to use AlterraLaniakea/jepa-guided-diffusion with Cosmos:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Fix base model: VJEPA 2.1 ViT-L/384
Browse files
README.md
CHANGED
|
@@ -9,7 +9,7 @@ tags:
|
|
| 9 |
- wts
|
| 10 |
- video-generation
|
| 11 |
base_model:
|
| 12 |
-
- facebook/vjepa2-vitl-fpc64-
|
| 13 |
- meta-llama/Llama-3.2-1B
|
| 14 |
- nvidia/Cosmos-Reason1-7B
|
| 15 |
pipeline_tag: other
|
|
@@ -32,7 +32,7 @@ Reason1 text encoder at inference time*.
|
|
| 32 |
|
| 33 |
| Part | Role | Source |
|
| 34 |
|------|------|--------|
|
| 35 |
-
| **X-Encoder** (frozen) |
|
| 36 |
| **Compressor** (trained) | BLIP-2-style **QFormer**: 512 query tokens, 4 layers, cross-attn every layer | in `model.pt` |
|
| 37 |
| **Backbone** (trained) | **last 4 layers of Llama-3.2-1B**, made bidirectional | arch from hub, weights in `model.pt` |
|
| 38 |
| **Head** (trained) | `output_projection` → `(512, 1024)` | in `model.pt` |
|
|
|
|
| 9 |
- wts
|
| 10 |
- video-generation
|
| 11 |
base_model:
|
| 12 |
+
- facebook/vjepa2-vitl-fpc64-384
|
| 13 |
- meta-llama/Llama-3.2-1B
|
| 14 |
- nvidia/Cosmos-Reason1-7B
|
| 15 |
pipeline_tag: other
|
|
|
|
| 32 |
|
| 33 |
| Part | Role | Source |
|
| 34 |
|------|------|--------|
|
| 35 |
+
| **X-Encoder** (frozen) | VJEPA 2.1 ViT-L/384 (`vjepa2_1_vit_large_384`), video mode → visual tokens | torch.hub (`facebookresearch/vjepa2`) |
|
| 36 |
| **Compressor** (trained) | BLIP-2-style **QFormer**: 512 query tokens, 4 layers, cross-attn every layer | in `model.pt` |
|
| 37 |
| **Backbone** (trained) | **last 4 layers of Llama-3.2-1B**, made bidirectional | arch from hub, weights in `model.pt` |
|
| 38 |
| **Head** (trained) | `output_projection` → `(512, 1024)` | in `model.pt` |
|