README: model card text only (remove repository-specific usage section)

MANIFEST.sha256

@@ -1,4 +1,4 @@
-bbd90f7c51e7d45fb8d5e206ac2f8c2b9b5e30bfc826be8c2da05f5acd15d588  README.md
+49dcca2d766497f3f28e8b1cb661228de3988a27fe04d71d2ec935e858f81e81  README.md
 3df4b53004db96437bb28b85f9bd4cf3447379def19f1a9dbbd77d830f184e75  model.pth
 d3f3401d5da979587382c29705b6eb260f41b452708b7da2b705bf8f5970c7ea  model.safetensors
 16451e080973c6c891307acbe148caa4367580c46e87d687dd23abeaea3f111b  model_config.yaml

README.md

@@ -223,64 +223,9 @@ A clip is one video-text pair. GPU compute is the mean `trtexec` GPU compute tim
 
 **Note:** The measured engine combines the vision and text towers in one graph, so every call runs both towers. Deployment can run faster with the towers separated: video clips are embedded once at indexing time with a vision-tower engine, and only the lightweight text tower runs for each query. Separate vision-tower and text-tower ONNX files are provided for this purpose. Separate-tower performance was not measured in PBR 309421.
 
-### Files in this repository
-
-Every file is covered by `MANIFEST.sha256` (`sha256sum -c MANIFEST.sha256`). The ONNX files and the two weight files encode the same model: the PyTorch towers rebuilt from `model.safetensors` reproduce the FP32 ONNX embeddings to within 6 × 10⁻⁷ (minimum cosine 0.99999994) and the FP16 ONNX embeddings to within 4 × 10⁻⁴ (minimum cosine 0.999998) on seeded and real-query probes, and `model.safetensors` equals `model.pth` after LoRA merge.
-
-| File | Bytes | SHA-256 (prefix) | Purpose |
-| --- | ---: | --- | --- |
-| `model.safetensors` | 1,491,073,676 | `d3f3401d5da97958…` | Fine-tuned weights with the vision LoRA adapters merged into the base linear layers (486 tensors, FP32; TAO `VideoCLIPPlModel` state-dict key names, `peft` disabled). Loads with `model_config.yaml`. |
-| `model_config.yaml` | 1,395 | `16451e080973c6c8…` | TAO `video_clip` model block for loading either weight file (see the header comments). |
-| `model.pth` | 1,493,513,435 | `3df4b53004db9643…` | TAO/Lightning checkpoint with the LoRA adapters unmerged (state dict only). Loads with `model_config.yaml` and `peft` enabled. |
-| `onnx/iv2clip_vision_fp32.onnx` | 3,908,049 | `2287e5d1845549e4…` | FP32 vision tower, ONNX opset 23, dynamic batch, LoRA merged; 96 fused `RMSNormalization` ops. Keep with its `.onnx.data` file. |
-| `onnx/iv2clip_vision_fp32.onnx.data` | 1,237,385,216 | `27e70bca3f0ca169…` | External weights of the FP32 vision tower. |
-| `onnx/iv2clip_text_fp32.onnx` | 1,227,330 | `26134b9cb78c0673…` | FP32 text tower, ONNX opset 23, dynamic batch. Keep with its `.onnx.data` file. |
-| `onnx/iv2clip_text_fp32.onnx.data` | 253,755,392 | `cacc6493c3a7863c…` | External weights of the FP32 text tower. |
-| `onnx/iv2clip_vision_fp16.onnx` | 619,141,817 | `141e726ae5dcf0bf…` | Vision tower for FP16 TensorRT deployment (ModelOpt AutoCast: FP16 bulk, `RMSNormalization` kept FP32, FP32 I/O, single file). Build strongly typed. |
-| `onnx/iv2clip_text_fp16.onnx` | 126,946,278 | `6e41fa1c7fb94e6b…` | Text tower for FP16 TensorRT deployment (AutoCast, FP32 I/O, single file). Build strongly typed. |
-| `onnx/tokenizer/` | 5,030,306 (5 files) | see `MANIFEST.sha256` | Hugging Face tokenizer equivalent to the model's CLIP BPE tokenizer (`openai/clip-vit-large-patch14`, MIT): `tokenizer.json`, `vocab.json`, `merges.txt`, `tokenizer_config.json`, `special_tokens_map.json`. |
-
-### Loading the weights
-
-The TAO `video_clip` model constructor needs the upstream base assets referenced in `model_config.yaml`: the InternVideo2 vision and alignment weights (`model.internvideo2clip_hf_id: OpenGVLab/InternVideo2_distillation_models`, resolved through the Hugging Face cache) and the Apple MobileCLIP-B-LT text-encoder file (`model.text_encoder`, `mobileclip_blt.pt`). Both are subject to their own licenses listed above. The adapter settings in `model_config.yaml` (`peft.vision`: `qkv` and `proj`, last 12 blocks, rank 8, alpha 16) match the released checkpoint.
-
-```python
-from omegaconf import OmegaConf
-from safetensors.torch import load_file
-from nvidia_tao_pytorch.config.video_clip.default_config import VideoCLIPExperimentConfig
-from nvidia_tao_pytorch.multimodal.video_clip.model.pl_video_clip_model import VideoCLIPPlModel
-
-cfg = OmegaConf.merge(OmegaConf.structured(VideoCLIPExperimentConfig), OmegaConf.load("model_config.yaml"))
-
-# merged weights (no LoRA modules)
-model = VideoCLIPPlModel(experiment_spec=cfg).eval()
-model.load_state_dict(load_file("model.safetensors"), strict=True)
-
-# or the checkpoint with the LoRA adapters still separate
-cfg.peft.enabled = True
-cfg.peft.vision.enabled = True
-model = VideoCLIPPlModel.load_from_checkpoint("model.pth", map_location="cpu", experiment_spec=cfg).eval()
-```
-
-### Using the ONNX towers
-
-- Vision tower: input `image` float32 `[B, 8, 3, 224, 224]`; output `image_embedding` float32 `[B, 512]`, L2-normalized. Sample 8 frames uniformly over the clip (`linspace(0, n-1, 8)`), RGB, bicubic resize to 224 × 224, scale to [0, 1], normalize with mean `(0.485, 0.456, 0.406)` and standard deviation `(0.229, 0.224, 0.225)`.
-- Text tower: inputs `input_ids` and `attention_mask`, both int64 `[B, 77]` (the mask is accepted for compatibility and ignored); output `text_embedding` float32 `[B, 512]`, L2-normalized. Tokenize with `onnx/tokenizer/` using `padding="max_length", truncation=True, max_length=77`.
-- The FP32 files also output the constants `logit_scale` (94.1502, already exponentiated) and `logit_bias` (0). Similarity is `logit_scale * (image_embedding · text_embedding) + logit_bias`; ranking only needs the dot product.
-- The opset-23 graphs use the fused `RMSNormalization` operator: onnxruntime 1.23 or newer is required. The FP16 files must be built **strongly typed** in TensorRT; a weakly typed `--fp16` build re-lowers the vision RMSNorm to half precision and collapses the vision embeddings.
-
-```bash
-trtexec --onnx=onnx/iv2clip_vision_fp16.onnx --stronglyTyped \
-  --minShapes=image:1x8x3x224x224 --optShapes=image:8x8x3x224x224 --maxShapes=image:16x8x3x224x224 \
-  --saveEngine=iv2clip_vision_fp16.plan
-trtexec --onnx=onnx/iv2clip_text_fp16.onnx --stronglyTyped \
-  --minShapes=input_ids:1x77,attention_mask:1x77 --optShapes=input_ids:8x77,attention_mask:8x77 \
-  --maxShapes=input_ids:16x77,attention_mask:16x77 --saveEngine=iv2clip_text_fp16.plan
-```
-
 ### How to use this model
 
-Use a TAO installation containing the `video_clip` InternVideo2-CLIP-L14 implementation, the matching base assets and tokenizer, and a complete experiment specification compatible with this checkpoint. The [TAO InternVideo2-CLIP overview](https://docs.nvidia.com/tao/tao-toolkit/latest/text/embedding/video_clip/overview.html) describes the model and data input format; the [training, evaluation, inference, and export guide](https://docs.nvidia.com/tao/tao-toolkit/latest/text/embedding/video_clip/training.html) documents the experiment specification and every subtask below. Set `model.type: internvideo2-clip-l14`, `model.num_frames: 8`, and `model.image_size: 224`, and restore the vision-LoRA configuration described above. The files in this repository are listed under **Files in this repository** above.
+Use a TAO installation containing the `video_clip` InternVideo2-CLIP-L14 implementation, the matching base assets and tokenizer, and a complete experiment specification compatible with this checkpoint. The [TAO InternVideo2-CLIP overview](https://docs.nvidia.com/tao/tao-toolkit/latest/text/embedding/video_clip/overview.html) describes the model and data input format; the [training, evaluation, inference, and export guide](https://docs.nvidia.com/tao/tao-toolkit/latest/text/embedding/video_clip/training.html) documents the experiment specification and every subtask below. Set `model.type: internvideo2-clip-l14`, `model.num_frames: 8`, and `model.image_size: 224`, and restore the vision-LoRA configuration described above. The Hugging Face repository provides the merged weights (`model.safetensors`), the TAO checkpoint with unmerged LoRA adapters (`model.pth`), the model configuration (`model_config.yaml`), the FP32 and FP16 ONNX towers with the tokenizer (`onnx/`), and a SHA-256 manifest.
 
 **Train** ([guide](https://docs.nvidia.com/tao/tao-toolkit/latest/text/embedding/video_clip/training.html#training-the-model)): fine-tune from the base weights with the recipe described under Model Architecture.