| license: other | |
| library_name: transformers | |
| tags: | |
| - multimodal | |
| - data-selection | |
| - llava | |
| - lora | |
| - clip | |
| # OnceSelect | |
| Pretrained artifacts for **OnceSelect**, a reusable data selector for multimodal instruction tuning. | |
| ## Contents | |
| - `selector_b32/selector.pt`: paper-default gated selector trained from frozen CLIP ViT-B/32 features. | |
| - `selector_b32/clustering.pt`: corresponding `K=128` source centroids and assignments. | |
| - `selector_b32/selector_training.json`: selector training and checkpoint-selection metadata. | |
| - `llava7b_lora_b32/`: best LLaVA-v1.5-7B LoRA trained on the subset selected by the default CLIP-B/32 selector. | |
| The selector uses a 3% centroid-nearest pseudo-labeled core, a 20% cluster-stratified validation split, and an intermediate checkpoint selected near a target validation macro accuracy of 0.60. The released checkpoint was selected at epoch 35 with validation macro accuracy 0.61078. | |
| The LLaVA adapter uses `lmsys/vicuna-7b-v1.5`, the LLaVA-v1.5 multimodal projector, and `openai/clip-vit-large-patch14-336` as its downstream vision tower. This downstream tower is distinct from the CLIP ViT-B/32 encoder used by the data selector. | |
| Core implementation and usage instructions are available in the OnceSelect GitHub repository. | |
| ## Integrity | |
| SHA-256 checksums are provided in `SHA256SUMS`. | |
| ## Citation | |
| Citation information will be added with the public paper release. | |