README.md

---
tags:
- multimodal
- routing
- uncertainty
- evidence-grounding
- sentence-transformers
- chinese
base_model: BAAI/bge-small-zh-v1.5
pipeline_tag: feature-extraction
---

# StreamSense-Serve 4.0 Router

This repository contains the trained, auditable routing artifact used by
[StreamSense-Serve 4.0](https://github.com/Jatshi/StreamSense-Serve). It is not a copy of the
Qwen or BGE base-model weights. The artifact stores four lightweight decision branches and their
jointly calibrated thresholds:

1. safety risk;
2. lightweight-evidence uncertainty;
3. cross-modal conflict;
4. visual dependency.

The runtime escalates to visual enhancement when **any** branch crosses its threshold. Risk and
visual dependency use frozen `BAAI/bge-small-zh-v1.5` embeddings followed by logistic heads;
uncertainty and conflict use physical signal and cross-modal features. The BGE revision is pinned
inside `router_artifact.json` and must be downloaded under its upstream terms.

## Files

- `router_artifact.json`: coefficients, normalizers, frozen encoder revision, four thresholds,
  training-data digest, and grouped-OOF metrics;
- `router_test_metrics.json`: results on 576 physical media cases from eight unseen semantic
  groups, including baselines, condition metrics, robustness slices, and group bootstrap;
- `e2e_summary.json`: formal 32-video × three-system result, including answer, citation, visual-call,
  and total-backend-call rates;
- `e2e_diagnostics.json`: group-bootstrap confidence intervals, paired system differences, and
  failure-mode counts;
- `SHA256SUMS`: integrity hashes for the published artifacts;
- `MODEL_CARD.md`: this card in a standalone filename for local archives.

## Measured result

| Metric | Value |
|---|---:|
| Development semantic groups | 24 |
| Grouped OOF folds | 4 |
| OOF escalation recall | 95.88% |
| Held-out semantic groups | 8 |
| Held-out physical cases | 576 |
| Held-out escalation recall | 94.81% |
| Held-out risk recall | 98.61% |
| Visual-enhancement rate | 94.97% |

The held-out split is deliberately difficult: 520/576 cases are oracle-positive, so even a
perfect router could avoid visual enhancement on at most 9.72% of this split. The measured 5.03%
reduction is not a production-traffic cost estimate.

The separate formal end-to-end run used 32 real MP4 cases sampled evenly from eight untouched
semantic groups and executed learned, always-visual, and never-visual systems once each (96 system
records). Quality pass rates were 65.63%, 68.75%, and 78.13%, respectively. The preregistered H5
hypothesis therefore failed: in this test, OCR already resolved the two visual groups and the
free-text VLM evidence introduced state inconsistencies instead of net quality. This negative
result is retained because it defines the next work item: non-OCR visual tasks and structured VLM
evidence. Latency values in this run are not comparable across systems because ASR/OCR outputs
were memoized in execution order.

## Reproduce

```bash
python -m pip install -e ".[semantic-router]"
python scripts/evaluate_v4_router.py \
  --features /path/to/features.jsonl \
  --router router_artifact.json \
  --output reproduced_metrics.json \
  --bootstrap-iterations 1000
```

The complete media builder, Faster-Whisper/RapidOCR extraction, group-isolated OOF training, and
end-to-end Qwen-VL/EvidenceAgent evaluation are in the GitHub repository. See the
[implementation status](https://github.com/Jatshi/StreamSense-Serve/blob/main/docs/V4_IMPLEMENTATION_STATUS_ZH.md),
[release notes and failure diary](https://github.com/Jatshi/StreamSense-Serve/blob/main/docs/V4_RELEASE_NOTES_ZH.md),
and [learning guide](https://github.com/Jatshi/StreamSense-Serve/blob/main/docs/V4_LEARNING_AND_INTERVIEW_ZH.md).

## Intended use and limitations

This artifact is an engineering benchmark router for evidence-grounded audio/video assistants.
It was trained on 32 generated Chinese semantic scenarios expanded into 2,304 physical corruption
conditions. It is not validated for medical decisions, surveillance, identity recognition, or
autonomous safety actions. The corpus uses one synthetic voice family and controlled visual
templates; accents, spontaneous speech, natural meetings, and deployment priors require separate
evaluation.

The endpoint experiment uses one text EvidenceAgent call per case and adds one image-bearing VLM
call only when the router escalates. Therefore `visual-enhancement rate` does not mean that all
non-escalated cases avoid every large-model call.