nvidia

/

InternVideo2-CLIP-anomaly-detection

Like 2

Follow

NVIDIA 72.3k

Adds the InternVideo2-CLIP-anomaly-detection release files: the TAO video_clip fine-tune of OpenGVLab InternVideo2-CLIP-L14 (vision LoRA, frozen MobileCLIP-B-LT text encoder) for text-to-video event and anomaly retrieval, released for non-commercial research under the NVIDIA Internal Scientific Research and Development Model License already set on this repository.

Contents

• model.safetensors — fine-tuned weights with the LoRA adapters merged (FP32, TAO state-dict key names) + model_config.yaml.
• model.pth — TAO checkpoint with the LoRA adapters unmerged (state dict only).
• onnx/ — FP32 vision and text towers (opset 23, dynamic batch, .onnx + .onnx.data), FP16 towers for strongly typed TensorRT builds, CLIP BPE tokenizer.
• README.md — model card, MANIFEST.sha256.

Verification before upload

• model.safetensors loads strict=True into a peft-disabled VideoCLIPPlModel; model.pth loads with load_from_checkpoint; both reproduce the reference embeddings exactly.
• FP32 ONNX towers vs the safetensors model on seeded and real-query probes: min cosine 0.99999994; FP16 towers within 1e-4.

Accuracy: PhysicalAI-Event-Videos test subset event-only retrieval mAP 63.12% / Hit@1 59.11% / Hit@5 87.04% (base 54.34 / 50.69 / 78.49); Kinetics-400 zero-shot Top-1 69.19% / Top-5 89.64%. Details in the README.

Repository stays private until the release review completes. Contact: Alice Li.