---
license: mit
base_model: facebook/vjepa2-vitl-fpc16-256-ssv2
tags:
  - core-ai
  - video-classification
  - video-understanding
  - vjepa2
  - apple-silicon
  - feature-extraction
library_name: coreai
pipeline_tag: video-classification
---

# V-JEPA 2 ViT-L (SSv2) — Core AI, **two-output** (logits + router embedding)

A Core AI export of [`facebook/vjepa2-vitl-fpc16-256-ssv2`](https://huggingface.co/facebook/vjepa2-vitl-fpc16-256-ssv2)
(375M, MIT) that returns **both** the SSv2 action logits **and the pooled `[1,1024]` router
embedding**.

```
pixel_values_videos [1,16,3,256,256] fp16
    ├─ logits     [1,174]    SSv2 action classes
    └─ embedding  [1,1024]   pooled attentive-pooler output
```

## Why two outputs

The stock export returns `logits` only. That is the wrong contract for anything using V-JEPA 2 as a
**content/router feature extractor** rather than an action classifier: the SSv2 checkpoint ships no
label names, its vocabulary is hand-object interactions, and it is semantically wrong for most
non-action domains. The `[1024]` pooled embedding is the reusable signal — feed it to a small
calibrated head for your own task.

Built for a video-optimization planner that fits a logistic probe over this embedding to pre-classify
content before an encode search.

## Measured

Apple **M5 Max** / 128 GB, fp16, GPU:

| | |
|---|---|
| Bundle | **708 MB** (675 MiB) |
| Load | **0.16 s** |
| Warm forward (16-frame clip) | **44 ms** |
| Engine logits vs torch oracle | cos **0.999964**, top-1 match |
| Engine embedding vs torch reference | cos **0.999940**, relL2 1.1e-02 (fp16 noise) |
| Recomposition `classifier(pooler(backbone(x)))` vs `.logits` | cos **1.0000000**, max\|Δ\| **0** — bit-exact |

⚠️ Latency is silicon-specific. Published M4 Max figures for the single-output bundle are 150–180 ms;
that is **different hardware**, not a difference between these bundles.

**Embedding separation** (6 signage clips, a semantic check no numeric gate performs): pairwise
cosine spans **0.146 → 0.987**, and the highest-similarity pair is the same content at two different
encodes (`_1080p` vs `_master`, cos 0.987). Content-identical inputs collapse together; unrelated
content separates.

## Preprocessing — on the host

The model does **not** normalize internally.

1. 16 frames, uniformly sampled across the clip
2. aspect-fill to 256×256, centre-crop (not a squash)
3. RGB, 0..1
4. ImageNet mean `[0.485, 0.456, 0.406]` / std `[0.229, 0.224, 0.225]`
5. `[1,16,3,256,256]`, fp16

## Use

```python
import asyncio, numpy as np, coreai.runtime as rt

async def main():
    opts = rt.SpecializationOptions.from_preferred_compute_unit_kind(rt.ComputeUnitKind.gpu())
    fn = (await rt.AIModel.load("vjepa2_ssv2_embed_fp16.aimodel", opts)).load_function("main")
    x = ...  # [1,16,3,256,256] fp16, preprocessed as above
    out = await fn(inputs={"pixel_values_videos": rt.NDArray(np.ascontiguousarray(x))})
    logits    = out["logits"].numpy()      # [1,174]
    embedding = out["embedding"].numpy()   # [1,1024]  ← the signal

asyncio.run(main())
```

## Reproduce

`export_fp16_embed.py` is included. It needs `coreai_models`, torch, and transformers ≥4.57, plus
oracles from the reference run (`oracle_*.npy` are included here, so the gate can be re-run without
recomputing them).

Two things in that script are load-bearing and easy to lose in a re-derivation:

- **`skip_predictor=True`.** `VJEPA2ForVideoClassification.forward` passes it; `VJEPA2Model` defaults
  it to `False`, which runs the entire JEPA *predictor* stack. `last_hidden_state` is
  `sequence_output` either way, so omitting it changes **no number** — every parity gate still
  passes, and the bundle silently carries a transformer stack nothing reads. **Parity gates certify
  values, never graph extent.**
- **The 3D-RoPE `squeeze(-1)` overlay.** The reference squeezes a non-1 dim (a torch no-op); the
  converter maps it to `ShrinkDims`, which requires size 1, and fails. Patched out, math unchanged.

The submodule resolution is discover-then-prove: it resolves `backbone`/`pooler`/`classifier` by
candidate name, then asserts the recomposition reproduces the model's own `.logits` before exporting.
If a future transformers renames them, it fails loudly instead of shipping a wrong "embedding".

## Files

| | |
|---|---|
| `vjepa2_ssv2_embed_fp16.aimodel/` | the bundle (`metadata.json`, `main.mlirb`, `main.hash`) |
| `export_fp16_embed.py` | the exact export that produced it |
| `oracle_input.npy` / `oracle_logits.npy` / `oracle_embedding.npy` | gate fixtures |

## License

MIT, inherited from Meta's V-JEPA 2.