covtoken — Label-Free Lesion-Subspace Token Economy for Medical Imaging

Code, gated evaluation, and a working paper draft for a label-free token-pruning method on frozen self-supervised medical vision transformers.

Reframed contribution (see gate_reports/SUMMARY.md). The load-bearing result is a label-free lesion subspace — a mid-layer geometry that localizes lesions WITHOUT labels across anatomy, modality (CT + ultrasound), and backbone (MedDINOv3 + DINOv2) — together with membership pruning (beats saliency pruning on small-lesion miss-rate), a conformal retention certificate, and lesion-routed depth (1.6× FLOPs). The original coverage-constrained optimization with an interpretable dual is reported as a clean negative result with a transferable mechanism (rank-based coverage rewards diverse spanning, rare pathology needs concentration). See gate_reports/NEGATIVE_RESULT.md.

Headline numbers

Mid-layer finding	lesion AUROC final-layer 0.565 → block-3 0.871
Cross-modality	density-A: lung CT 0.87, kidney 0.82, breast US 0.73 (DINOv2; attention 0.49)
Membership > saliency pruning	LIDC +27.6/+15.8, KiTS23 +7.4, BUSI +13.8/+19.0 pts
Conformal retention cert.	empirical coverage 0.978 ≥ nominal 0.90
Lesion-routed depth	1.6× FLOPs at 98% small-lesion sensitivity
Negative result	coverage floor 0.22 vs membership 0.82 (small-lesion recall)

How the study was run, and what failed

Question. Where does a frozen CT foundation model (MedDINOv3, ViT-B/16) store lesion information, and can a model spend fewer tokens on easy slices without losing lesions?

Compute path. One person, no institutional cluster. LIDC-IDRI DICOM (about 124 GB) was streamed from bucket storage into cloud jobs, never downloaded locally. Frozen embeddings were encoded once into a cached token bank (2.1M CT tokens), and every probe ran on that cache instead of fine-tuning the backbone. That gave up end-to-end adaptation, but made dozens of controlled probes affordable.

Gated protocol. Seven gates (0–6), each with pass/fail thresholds fixed before the run (configs/thresholds.lock.json), and a report written after each (gate_reports/). Several gates failed, and the failures are kept:

• Gate 1 failed at the final layer. The label-free lesion scores reached AUROC 0.565 against 0.767 for attention saliency (gate_reports/gate_1_postmortem.md). The signal was there at mid-depth: block 3 reached 0.871, which passed the locked threshold.
• Coverage-constrained token selection failed. Small-lesion recall 0.22 against 0.82 for membership pruning (gate_reports/NEGATIVE_RESULT.md): coverage rewards diverse spanning, but rare pathology needs concentration.
• The low-rank hypothesis was refuted. A spectrum analysis ruled out redundancy: lesion tokens matter because they are rare (research_v4/rarity_route_derivation.md).
• The mid-layer peak tracks view-invariance (ρ = −0.94 with AUROC; research_v3/SUMMARY.md), not loss of spatial information.
• Gate 5 was not run. Whether per-view token selection during pretraining would break the self-supervised invariance is untested; the method takes the gate's fallback and is applied at inference time only.

A data error found and fixed. The original manifest's per-slice nodule positions did not line up with the authoritative DICOM-SEG masks under any slice ordering. All patch masks were rebuilt from DICOM-SEG (jobs/materialize_lidc_masks_job.py) before any gate result was trusted. Gate results are scored on held-out data with 95% confidence intervals.

Layout

subspace/     label-free lesion subspace: density (A) + residual (B) constructions
coverage/     rankme / coding-rate / energy functionals (the FALSIFIED coverage objective)
gate/         constrained pruner (Gumbel mask + dual) + per-image certificate  [negative result]
arch/         conformal_head, routed_depth, volumetric  (Phase-6 components)
backbone/     frozen MedDINOv3 ViT-B/16 loader (DINOv2 used for ultrasound, in jobs/)
data/         CT token-bank builder, eval-only mask loading + label-leak guard
eval/         DeLong / paired-bootstrap / Spearman stats; gate runners
jobs/         all experiments as Hugging Face Jobs (materialize, banks, gates, ablations)
gate_reports/ machine-readable per-gate decision records + SUMMARY + NEGATIVE_RESULT
configs/      thresholds.lock.json (Phase-1b calibrated, locked)
paper/        working_draft.md + figures/ + venue_notes.md
tests/        CI label-leak test (no label may touch subspace construction)

Reproducibility

All experiments ran as Hugging Face Jobs. Backbones: ricklisz123/MedDINOv3-ViTB-16-CT-3M (CT), facebook/dinov2-base (ultrasound). Datasets: LIDC-IDRI, KiTS23, MSD Task03 Liver, MSD Task07 Pancreas, BUSI. Masks are evaluation-only; tests/test_label_leak.py fails the build if a label reaches subspace construction. The frozen DINOv3 model code is installed from github.com/facebookresearch/dinov3 at job time (not vendored here).

Decision records: gate_reports/. Locked thresholds: configs/thresholds.lock.json.