Commit History
-
Add scope section: full Video-MME mini 2700Q result (+0.22 pp)
Verified signatureFollowing the 300Q release, the eval was extended to the full 2700Q split. Overall Δ +0.22 pp. README adds: (1) Scope note callout under TL;DR, (2) updated Caveat in Reproducibility section pointing to (3) new 'Scope on the full Video-MME mini (2700Q)' section. The original 300Q numbers are unchanged and remain reproducible by recipe; this addition characterizes the design envelope (short-clip, low-frame-budget) on the full balanced split.d0f5738
-
Add --start-idx for chunked evaluation (MPS resilience)
Verified signatureApple MPS can corrupt accelerator state on long Qwen3-VL inference runs (~800+ questions), producing AcceleratorError. Add --start-idx K so callers can skip the first K filtered questions, enabling a chunked-runner pattern: each chunk is a fresh Python process with clean MPS state. Also wrap the per-question inference in try/except: on AcceleratorError or any other inference failure, save partial results and exit gracefully so an outer chunked-runner can resume from start-idx + completed_count.50d1d87 -
example_usage: rename Wild-mode → MCQ-mode terminology
Verified signature7df1847 -
eval_videomme: rename --mode wild → mcq (wild kept as deprecated alias); update docstrings to use MCQ-mode terminology
Verified signature3081b91 -
Wording hotfix: scope clarity + MCQ-mode terminology
Verified signatureFollowing audit, retire 'wild mode' / 'in-the-wild deployment' wording in favor of: - 'MCQ mode (no task_type)' for the default 64.3% setting - 'Task-aware MCQ mode' for the 66.3% setting (uses dataset task taxonomy) Add explicit Scope section: this release evaluates query-aware frame selection for video MCQ / decision-style QA. The selector may use the question + answer options as its CLIP query. This is appropriate for Video-MME-style benchmarks and operational triage workflows where the system chooses among predefined actions/alert categories. Not an open-ended video understanding claim. Add Motivation note: started from CCTV/video-security R&D under tight frame-budget constraints. The artifact is general-purpose, not a product-specific CCTV model. Method, numbers (64.3% / 66.3%), and Qwen3-VL attribution unchanged.c04d819 -
Fix transformers 5.x API change: get_text_features now returns BaseModelOutputWithPooling
Verified signatureIn transformers >=5.0, CLIPModel.get_text_features and get_image_features return BaseModelOutputWithPooling instead of a Tensor. The projected text/image embedding lives at .pooler_output (after projection). Add backward-compatible accessor that works on both old (tensor) and new (output object) return types.5e31798 -
Polish: distinguish wild-mode (~44%) vs benchmark-mode (~56%) gap recovery
Verified signaturePer auditor feedback: the previous '56% of the gap' line was the benchmark-mode number; spell out both modes' recovery fractions to avoid ambiguity for readers who skim the lede.7cb17d8 -
Add Acknowledgements / Related Work section
Verified signatureAcknowledge that query-aware and adaptive frame selection for Video-LLMs is an active research direction; clarify that this release is an independent engineering implementation focused on small-model, low-frame-budget video QA and CCTV-style deployment.97b9bd0 -
Initial release: DW-KhotTaeVL-2B-QueryFrames v1.0
Verified signatureQuery-aware frame selection wrapper for Qwen3-VL-2B-Instruct. Wild mode: 64.3% on Video-MME mini 300Q (+7.3pp vs stock 57.0%). Benchmark mode: 66.3% (+9.3pp), 12/12 task buckets non-negative. Zero trainable parameters, no model weights modified. Built on Qwen/Qwen3-VL-2B-Instruct (Apache 2.0). Frame scorer: openai/clip-vit-large-patch14 (MIT). Author: Deaw (HF: @commandeaw).84c8a9d -
initial commit
Verified signature9bf418b