README.md

metadata

license: other
library_name: transformers
tags:
  - multimodal
  - data-selection
  - llava
  - lora
  - clip

OnceSelect

Pretrained artifacts for OnceSelect, a reusable data selector for multimodal instruction tuning.

Contents

• selector_b32/selector.pt: paper-default gated selector trained from frozen CLIP ViT-B/32 features.
• selector_b32/clustering.pt: corresponding K=128 source centroids and assignments.
• selector_b32/selector_training.json: selector training and checkpoint-selection metadata.
• llava7b_lora_b32/: best LLaVA-v1.5-7B LoRA trained on the subset selected by the default CLIP-B/32 selector.

The selector uses a 3% centroid-nearest pseudo-labeled core, a 20% cluster-stratified validation split, and an intermediate checkpoint selected near a target validation macro accuracy of 0.60. The released checkpoint was selected at epoch 35 with validation macro accuracy 0.61078.

The LLaVA adapter uses lmsys/vicuna-7b-v1.5, the LLaVA-v1.5 multimodal projector, and openai/clip-vit-large-patch14-336 as its downstream vision tower. This downstream tower is distinct from the CLIP ViT-B/32 encoder used by the data selector.

Core implementation and usage instructions are available in the OnceSelect GitHub repository.

Integrity

SHA-256 checksums are provided in SHA256SUMS.

Citation

Citation information will be added with the public paper release.