| --- |
| language: en |
| library_name: mctr |
| tags: |
| - multi-camera-tracking |
| - multi-object-tracking |
| - mmptracking |
| - video |
| - checkpoint |
| pipeline_tag: other |
| license: bsd-3-clause |
| pretty_name: MCTR MMPTracking checkpoints |
| --- |
| |
| # MCTR — Multi Camera Tracking Transformer (MMPTracking checkpoints) |
|
|
| Checkpoints that reproduce the MMPTrack validation results of |
|
|
| > **MCTR: Multi Camera Tracking Transformer**, Alexandru Niculescu-Mizil, Deep Patel, Iain Melvin. |
| > [arXiv:2408.13243](https://arxiv.org/abs/2408.13243) · [code](https://github.com/necla-ml/mctr) |
|
|
| MCTR is an end-to-end multi-camera multi-object tracker: a DETR-style detector per camera |
| view, a shared set of **track embeddings** updated every frame, and soft probabilistic |
| track↔detection association trained with differentiable losses. |
| A single finetuned checkpoint per environment serves both reported variants: |
|
|
| - **MCTR** — outputs the per-view detection boxes (`scripts/trackeval_mmptrack.py`) |
| - **MCTR-TB** — outputs the track-head predicted boxes (`scripts/trackeval_trackbox_mmptrack.py`) |
|
|
| Each checkpoint is finetuned on one environment (fixed number of cameras / clips), |
| so there is one model per scene: cafe, industry, lobby, office, retail. |
|
|
| ## Contents |
|
|
| | Scene | Cameras | File | Size (GB) | SHA-256 (first 16) | |
| |-------|---------|------|-----------|--------------------| |
| | cafe | 4 | `cafe/mctr_cafe_epoch99.pth` | 0.23 | `069d9a899211d902` | |
| | industry | 4 | `industry/mctr_industry_epoch99.pth` | 0.23 | `7f59c4e85afadad6` | |
| | lobby | 4 | `lobby/mctr_lobby_epoch99.pth` | 0.23 | `4421777c62f7c076` | |
| | office | 5 | `office/mctr_office_epoch99.pth` | 0.25 | `36db4869e9246262` | |
| | retail | 6 | `retail/mctr_retail_epoch99.pth` | 0.26 | `f5ba93e1c4a5085f` | |
|
|
| Each scene folder contains: |
|
|
| - `mctr_<scene>_epoch99.pth` — the finetuned checkpoint (epoch 99 of a 100-epoch finetune |
| of the 2-stage training protocol: `pairwise_init.yaml` → `pairwise.yaml`) |
| - `train_config.yaml` — exact training config of the run |
| - `metrics.json` — provenance + metrics (paper reference and reproduction) |
|
|
| The `.pth` is a `torch.save` dict with keys `cfg` (yacs config of the run), `state_dict` |
| (the `PAIRWISE` model, already de-`module.`-prefixed), `loss`, `epoch`. It is loaded by |
| `main_pairwise._build_model` in the training repo with `strict=True`; the camera count and |
| clip set are fixed by the embedded config. |
|
|
| ## Usage |
|
|
| ```sh |
| git clone https://github.com/necla-ml/mctr && cd mctr && make pull |
| mamba env create -f mcmot39 # conda env from the repo |
| mamba activate mcmot39 # conda activate mcmot39 |
| |
| # point the eval scripts at the MMPTracking dataset root you use, then: |
| python scripts/trackeval_mmptrack.py /path/to/mctr_cafe_epoch99.pth # MCTR |
| python scripts/trackeval_trackbox_mmptrack.py /path/to/mctr_cafe_epoch99.pth # MCTR-TB |
| ``` |
|
|
| Notes: |
| - The eval scripts hardcode the dataset root `/net/mlfs02/data/projects/shared/datasets/MMPTracking/` |
| and the MMPTracking clip layout (per-scene folders, `64pm` subsample); edit |
| `cfg.DATASET.ROOT` in the scripts to match your copy of the data. |
| - The scripts write tracklet files under `scripts/eval_outputs/` and call |
| `submodules/trackeval/scripts/run_mot_challenge.py` (HOTA/CLEAR/Identity, no preproc). |
| - Inference is online, frame-by-frame, batch size 1 (`keep_prob=0.9`); ~233 MB model, |
| roughly linear cost in the number of cameras. |
|
|
|
|
