Point to current paper and 80 main-table router weights

README.md

@@ -1,67 +1,32 @@
 ---
-language:
-- en
+license: mit
 tags:
-- computer-vision
-- model-routing
-- reproducibility
+- model-collaboration
+- learning-to-defer
+- vision-language
 ---
 
-# ModelCollaboration: five-seed results, router weights, and paper
+# Learned Routing for Specialist–VLM Collaboration
 
-**Current release: 13 September 2026, paper revision 32.** The current method uses four-state cross-entropy training and the routing score **P(rescue) − P(harm)**. It was selected by task-balanced validation Score25 **mean − sample standard deviation** across seeds **2026–2030**. Current main results average all five runs; the full objective comparison contains **5 methods × 16 model pairs × 5 seeds = 400 router fits**. The 80 main-method routers are a subset of those 400.
+Artifacts for **When Should a Vision Specialist Defer? Budgeted Collaboration with Vision–Language Models**.
 
-The current replay and experiment sources are shipped **in this Hugging Face repository**. The [original GitHub repository](https://github.com/nohi191212/ModelCollaboration) retains the earlier implementation and setup documentation; its original single-seed result tables and replay entry are not the current five-seed release.
+[Code and reproduction instructions](https://github.com/nohi191212/ModelCollaboration) · [Latest paper](releases/20260913_lr/main.pdf) · [Supplement](releases/20260913_lr/appendix.pdf)
 
-## Current files
+## Latest release
 
-| File or directory | Contents |
-|---|---|
-| `router_weights_five_seeds.tar.gz` | All 400 router checkpoints, run configurations, and training-set normalization parameters; extracts into `router_weights_five_seeds/runs/`. |
-| `experiment_data_five_seeds.tar.gz` | All 400 validation/test raw router-logit records, exact float32 routing scores, configurations and evaluation evidence, plus shared per-example task outcomes and sample identifiers for the 16 model pairs; extracts into `experiment_data/`. |
-| `scripts/replay_five_seed_results.py` | Current NumPy-only replay: recomputes scores, validation-selected operating points, curves, five-seed means/SDs, and objective selection from the released evidence. |
-| `paper_results/` | Revision-32 data, source tables, figures, and plotting sources. `data/results.json` is the current main-table/curve summary. `data/method_lock_five_seeds.json` records the final objective-selection rule and values. |
-| `main.pdf`, `appendix.pdf` | Current paper and full supplementary material, including the complete validation/test objective comparison. |
-| `reproduction/` | Training, evaluation, and five-seed selection sources with explicit local paths and safe import behavior. These sources require the original prepared feature caches for retraining; those caches are not part of the compact replay archive. |
-| `docs/` | [Five-seed protocol](docs/FIVE_SEED_PROTOCOL.md), [artifact and source guide](docs/ARTIFACTS.md), and [release changes](docs/RELEASE_NOTES.md). |
+Use [releases/20260913_lr](https://huggingface.co/nohi191212/ModelCollaboration/tree/main/releases/20260913_lr).
 
-## Recompute the current results without running any models
+- **80 main-table router checkpoints:** 16 model pairs × five seeds, in `full_budget_20260913_weights.tar.gz`.
+- Experiment code, configurations, histories and per-example outputs, including the other objectives and scoring experiments. Only the main-table router weights are included in this release.
+- Eight specialist input caches covering train, validation and test splits, with an input manifest.
+- Paper sources and numerical summaries in the code repository; latest PDFs and result data here.
 
-Download the current small files and the new evidence archive into one directory. The router weights are **not required** for this numerical replay.
+To reconstruct Tables 1 and 3 without training, download `paper_replay_runs.tar.gz` and `paired_outcomes.tar.gz`, then follow the [reproduction instructions](https://github.com/nohi191212/ModelCollaboration/blob/main/docs/REPRODUCE.md). The replay checks 80 main runs, 320 Table 3 runs and 19,392 curve points.
 
-```bash
-pip install huggingface_hub numpy
-hf download nohi191212/ModelCollaboration --local-dir modelcollaboration \
-  --include "README.md" "docs/*" "scripts/replay_five_seed_results.py" \
-  "experiment_data_five_seeds.tar.gz" "paper_results/*" "main.pdf" "appendix.pdf"
-cd modelcollaboration
-tar -xzf experiment_data_five_seeds.tar.gz
-python scripts/replay_five_seed_results.py --artifacts . --output-dir replayed
-```
+Main checkpoints use validation selection across 0–100% calls. Table 3 preserves its earlier checkpoint selection settings, detailed in the supplement.
 
-The replay writes `replayed_main_five_seeds.csv`, `replayed_objectives_five_seeds.csv`, `replayed_per_seed.csv`, `replayed_same_checkpoint_controls.csv`, and `verification_five_seeds.json`. It does not execute the specialist, VLM, or router. It independently checks the score transformations from raw logits, then uses saved float32 scores to preserve the original threshold-boundary decisions when recomputing task metrics. ConstructionSite is evaluated by event macro-F1; the other tasks use their saved correctness outcomes. The [artifact guide](docs/ARTIFACTS.md) explains the precision check.
+Root-level archives are retained for earlier releases. The latest release does not add supplementary router checkpoints.
 
-The release verification passed for all **400 fits**, **160 same-checkpoint scoring controls**, **118,160 learned-routing curve points**, and **3,232 confidence-routing curve points**, including the current main table and five-seed means/sample SDs. Confidence scores and validity flags are included for independent replay of that baseline. See [`verification_five_seeds.json`](verification_five_seeds.json) and [`replayed/`](https://huggingface.co/nohi191212/ModelCollaboration/tree/main/replayed) for the verified outputs.
+## Data and licenses
 
-Validation thresholds are transferred unchanged to test samples. A deployed threshold can decide for each arriving sample; it does not require access to all test inputs. Exact-budget test rankings are reported separately as batch diagnostics. See the [protocol](docs/FIVE_SEED_PROTOCOL.md) for this distinction and the documented earlier test inspection.
-
-## Download the trained routers
-
-```bash
-hf download nohi191212/ModelCollaboration router_weights_five_seeds.tar.gz --local-dir .
-tar -xzf router_weights_five_seeds.tar.gz
-```
-
-Current main-method directories end in `four_difference_s2026` through `four_difference_s2030`. The other directories contain the four matched objective/scoring controls. A checkpoint must be used together with its configuration, training normalization, and corresponding frozen feature encoders. Per-run thresholds belong to that run; an average threshold is not a replacement for the five individual policies.
-
-## What changed in the evidence
-
-At the validation-selected test operating points, the current five-seed means outperform confidence routing in **13/16** pairs, use fewer VLM calls in **11/16**, achieve both in **10/16**, and outperform both stand-alone models in **8/16**. These are comparisons of means, not a claim that every difference is statistically significant. Standard deviations describe router-training randomness with frozen encoders, specialist outputs, and VLM outputs.
-
-For the two cost examples in Fig. 3, InstanceVG/MiniCPM improves the metric by **6.15 percentage points**, with **70.8% fewer estimated FLOPs** and **68.6% less estimated serial time** than confidence routing. YOLO26x/Qwen improves by **3.90 points**, with reductions of **53.0%** and **52.5%**. Costs include the frozen routing encoders and routing head on every input. They reuse measured A800 component costs (64 inputs, batch one) and scale VLM cost by the actual call rate; they are not a newly measured full-system deployment trace. Savings are pair-dependent: the current InstanceVG/Qwen selected policy has a high call rate and does not retain the previous single-seed saving claim.
-
-## Earlier archives and external resources
-
-`weights.tar`, `experiment_data.tar.gz`, `replayed_main.csv`, and the earlier verification records are **legacy single-seed artifacts**. They must not be used to reconstruct the current five-seed tables. The unchanged distilled encoders and trained construction specialists in `weights.tar` are still reusable: `encoders/8M/` and `specialists/` remain relevant, while its `main/runs/` contains the old 16 routers.
-
-Original images, upstream VLM weights, some specialist weights, and full prepared feature caches remain external. Obtain the original resources and preparation instructions from the [GitHub setup documentation](https://github.com/nohi191212/ModelCollaboration/tree/main/docs), then use the current scripts in `reproduction/` for the new objective study. This release supports compact numerical replay without large-model inference; it does not claim one-command reproduction from raw images. Original dataset/model licenses and access conditions continue to apply; this model card grants no new license to third-party assets.
+Original code is MIT licensed. Upstream model and dataset licenses remain unchanged. This release provides derived experiment features, labels and outputs; obtain original images from their providers. ConstructionSite-derived data retain CC BY-NC 4.0 terms. See [model and dataset sources](https://github.com/nohi191212/ModelCollaboration/blob/main/docs/MODELS_AND_DATA.md).