File size: 1,406 Bytes
56e6aae | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | ---
license: other
library_name: transformers
tags:
- multimodal
- data-selection
- llava
- lora
- clip
---
# OnceSelect
Pretrained artifacts for **OnceSelect**, a reusable data selector for multimodal instruction tuning.
## Contents
- `selector_b32/selector.pt`: paper-default gated selector trained from frozen CLIP ViT-B/32 features.
- `selector_b32/clustering.pt`: corresponding `K=128` source centroids and assignments.
- `selector_b32/selector_training.json`: selector training and checkpoint-selection metadata.
- `llava7b_lora_b32/`: best LLaVA-v1.5-7B LoRA trained on the subset selected by the default CLIP-B/32 selector.
The selector uses a 3% centroid-nearest pseudo-labeled core, a 20% cluster-stratified validation split, and an intermediate checkpoint selected near a target validation macro accuracy of 0.60. The released checkpoint was selected at epoch 35 with validation macro accuracy 0.61078.
The LLaVA adapter uses `lmsys/vicuna-7b-v1.5`, the LLaVA-v1.5 multimodal projector, and `openai/clip-vit-large-patch14-336` as its downstream vision tower. This downstream tower is distinct from the CLIP ViT-B/32 encoder used by the data selector.
Core implementation and usage instructions are available in the OnceSelect GitHub repository.
## Integrity
SHA-256 checksums are provided in `SHA256SUMS`.
## Citation
Citation information will be added with the public paper release.
|