nvidia

/

InternVideo2-CLIP-anomaly-detection

Like 7

Follow

NVIDIA 72.6k

Adds the InternVideo2-CLIP-anomaly-detection v1.0 release files: the TAO video_clip fine-tune of OpenGVLab InternVideo2-CLIP-L14 (vision LoRA on the last 12 blocks, frozen MobileCLIP-B-LT text encoder, preservation regularization) trained on the PhysicalAI-Event-Videos train subset, released for non-commercial research under the NVIDIA Internal Scientific Research and Development Model License already set on this repository.

Contents (19 files, 4.9 GB)

• model.safetensors — fine-tuned weights with the 24 LoRA modules merged (486 tensors, 372,752,643 parameters, FP32, TAO state-dict key names) + model_config.yaml.
• model_epoch_002_step_04011.no_teacher.pth — TAO checkpoint, LoRA unmerged, teacher/optimizer state removed + train_experiment.yaml.
• onnx_opset23/ — FP32 vision and text towers (opset 23, dynamic batch, .onnx + .onnx.data), ModelOpt AutoCast FP16 towers for strongly typed TensorRT builds, deploy config, export spec, CLIP BPE tokenizer.
• README.md — model card (research release), MANIFEST.sha256.

Verification before upload

• model.safetensors loads strict=True into a peft-disabled VideoCLIPPlModel and reproduces the merged-checkpoint reference embeddings exactly (max abs diff 0).
• FP32 ONNX towers vs the safetensors model on seeded and real-query probes: min cosine 0.99999994, max abs diff 5.7e-7.
• Checkpoint and ONNX SHA-256 values equal the internal v1.0 release manifest.

Accuracy: PhysicalAI-Event-Videos test subset event-only retrieval mAP 63.12% / Hit@1 59.11% / Hit@5 87.04% (base 54.34 / 50.69 / 78.49); Kinetics-400 zero-shot Top-1 69.19% / Top-5 89.64%. Details in the README.

Repository stays private until the release review completes. Contact: Alice Li (alicli@nvidia.com).

Superseded by a new pull request.