Hugging Face's logo

nohi191212
/
ModelCollaboration

model-collaboration
learning-to-defer
vision-language
Model card Files
xet
 Community
ModelCollaboration / docs /RELEASE_NOTES.md
nohi191212's picture
nohi191212
Publish five-seed probability-difference results, 400 routers, and revision 32
466c2b6 verified
preview code
|
Raw History Blame Contribute Delete
2.73 kB

Release changes — 13 September 2026

This release synchronizes the Hugging Face artifacts with paper revision 32. Existing legacy archives remain available for historical checks. The original GitHub repository is not modified by this release.

New current artifacts

  • 400 router checkpoints, configurations, and normalization statistics across five objectives, sixteen pairs, and five seeds.
  • Raw validation/test router logits and per-example outcome evidence for recomputing five-seed results, including the 80 current main-method routers and same-checkpoint score controls.
  • A current NumPy-only replay script hosted here, with five-seed main/objective outputs and verification.
  • Current main/supplementary PDFs, tables, curves, individual-seed summaries, sample SDs, and the final validation mean-minus-SD selection record.
  • Portable experiment-source entries and explicit distinction between compact numerical replay and retraining from full prepared caches.

Results and interpretation updated

The main routing score changes from the earlier probability ratio to probability difference. All current learned main-table values are five-seed averages. Confidence and stand-alone predictions remain fixed. The objective table includes probability difference and rescue classification; its full validation/test version is in the supplement.

Current counts are 13/16 above confidence, 11/16 with fewer calls, 10/16 meeting both, and 8/16 above both stand-alone models. The last count corrects a floating-point tie in ResNet/Qwen: two rescued and two harmed examples across five seeds have zero net gain. No per-run metric was altered to make this correction.

Fig. 2 displays GLSim/MiniCPM, InstanceVG/MiniCPM, ViLT/Qwen, and YOLO26x/Qwen. Fig. 3 uses InstanceVG/MiniCPM and YOLO26x/Qwen, with frozen-encoder and router-head overhead included for every input. Old InstanceVG/Qwen cost-saving claims do not describe its current high-call policy.

The main claim concerns useful routing from pre-call information and training experience in suitable specialist/VLM settings. It does not require four-state targets to dominate all simpler controls. Mean comparisons are not blanket significance claims. Five-seed deviations cover router training only, and the earlier test-access history remains documented.

Legacy files

The original weights.tar, experiment_data.tar.gz, replayed_main.csv, and earlier verification files describe the prior single-seed release. Only the unchanged frozen encoders and construction specialist portions of the original weights archive are reused by the new study. For the current tables use the new archives and scripts/replay_five_seed_results.py named in the main README.