---
language: en
library_name: mctr
tags:
- multi-camera-tracking
- multi-object-tracking
- mmptracking
- video
- checkpoint
pipeline_tag: other
license: bsd-3-clause
pretty_name: MCTR MMPTracking checkpoints
---

# MCTR — Multi Camera Tracking Transformer (MMPTracking checkpoints)

Checkpoints that reproduce the MMPTrack validation results of

> **MCTR: Multi Camera Tracking Transformer**, Alexandru Niculescu-Mizil, Deep Patel, Iain Melvin.
> [arXiv:2408.13243](https://arxiv.org/abs/2408.13243) · [code](https://github.com/necla-ml/mctr)

MCTR is an end-to-end multi-camera multi-object tracker: a DETR-style detector per camera
view, a shared set of **track embeddings** updated every frame, and soft probabilistic
track↔detection association trained with differentiable losses.
A single finetuned checkpoint per environment serves both reported variants:

- **MCTR** — outputs the per-view detection boxes (`scripts/trackeval_mmptrack.py`)
- **MCTR-TB** — outputs the track-head predicted boxes (`scripts/trackeval_trackbox_mmptrack.py`)

Each checkpoint is finetuned on one environment (fixed number of cameras / clips),
so there is one model per scene: cafe, industry, lobby, office, retail.

## Contents

| Scene | Cameras | File | Size (GB) | SHA-256 (first 16) |
|-------|---------|------|-----------|--------------------|
| cafe | 4 | `cafe/mctr_cafe_epoch99.pth` | 0.23 | `069d9a899211d902` |
| industry | 4 | `industry/mctr_industry_epoch99.pth` | 0.23 | `7f59c4e85afadad6` |
| lobby | 4 | `lobby/mctr_lobby_epoch99.pth` | 0.23 | `4421777c62f7c076` |
| office | 5 | `office/mctr_office_epoch99.pth` | 0.25 | `36db4869e9246262` |
| retail | 6 | `retail/mctr_retail_epoch99.pth` | 0.26 | `f5ba93e1c4a5085f` |

Each scene folder contains:

- `mctr_<scene>_epoch99.pth` — the finetuned checkpoint (epoch 99 of a 100-epoch finetune
  of the 2-stage training protocol: `pairwise_init.yaml` → `pairwise.yaml`)
- `train_config.yaml` — exact training config of the run
- `metrics.json` — provenance + metrics (paper reference and reproduction)

The `.pth` is a `torch.save` dict with keys `cfg` (yacs config of the run), `state_dict`
(the `PAIRWISE` model, already de-`module.`-prefixed), `loss`, `epoch`. It is loaded by
`main_pairwise._build_model` in the training repo with `strict=True`; the camera count and
clip set are fixed by the embedded config.

## Usage

```sh
git clone https://github.com/necla-ml/mctr && cd mctr && make pull
mamba env create -f mcmot39        # conda env from the repo
mamba activate mcmot39             # conda activate mcmot39

# point the eval scripts at the MMPTracking dataset root you use, then:
python scripts/trackeval_mmptrack.py          /path/to/mctr_cafe_epoch99.pth   # MCTR
python scripts/trackeval_trackbox_mmptrack.py /path/to/mctr_cafe_epoch99.pth   # MCTR-TB
```

Notes:
- The eval scripts hardcode the dataset root `/net/mlfs02/data/projects/shared/datasets/MMPTracking/`
  and the MMPTracking clip layout (per-scene folders, `64pm` subsample); edit
  `cfg.DATASET.ROOT` in the scripts to match your copy of the data.
- The scripts write tracklet files under `scripts/eval_outputs/` and call
  `submodules/trackeval/scripts/run_mot_challenge.py` (HOTA/CLEAR/Identity, no preproc).
- Inference is online, frame-by-frame, batch size 1 (`keep_prob=0.9`); ~233 MB model,
  roughly linear cost in the number of cameras.