Charansaiponnada/video-query-router

Video Query Router

Reads a question about a video library and says where to search: the whole video, a clip, or a single frame. Small (0.43 MB), fast (about 2 ms per query on a CPU), no GPU.

Type	Needs	Example
video	the full video: topic, summary, "which is NOT shown", order of many events	What is the main idea of the video?
clip	a few seconds to minutes: an action, an event, something said	What does the man do after swimming?
frame	one picture: colour, clothes, on-screen text, a score	What colour is the news presenter's coat?

Model: logistic regression (scikit-learn) on words and word pairs, letter chunks, and 7 hand-made cue flags. If confidence is below 0.8, the router also searches the second most likely index.

Use

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("Charansaiponnada/video-query-router", local_dir="video-query-router")
sys.path.insert(0, path)  # the pickle needs the small intent_router package stored next to it
from intent_router.router import Router

router = Router(f"{path}/model.joblib")
print(router.route(["what colour is the car at 2:15?"]))
# [{'query': ..., 'type': 'frame', 'confidence': ..., 'search': ['frame_index']}]

Needs scikit-learn and joblib; trained with scikit-learn 1.7. model.joblib is a pickle: only load it from a source you trust.

Results (Video-MME 10% sample, 270 questions, 5-fold cross-validation split by video)

Model	Accuracy [95% range]	Right index searched	p50 ms	Size
Keyword rules only	66.7% [61.1, 71.9]	66.7%	0.06	~0
This model	70.0% [64.8, 75.6]	83.0%	1.9	0.43 MB
MiniLM sentence encoder + cue flags	72.6% [67.8, 77.8]	84.1%	13.7	91 MB

Training data: the 270 Video-MME questions (labeled by hand with one written rule) and 120 hand-written search-style queries. Limits: one labeler, small data, English only.