SichengLu commited on
Commit
6f1ce8a
·
verified ·
1 Parent(s): 29a72fe

Polish the model card: badges, hero figure, and a tighter structure

Browse files
Files changed (1) hide show
  1. README.md +48 -56
README.md CHANGED
@@ -12,28 +12,29 @@ library_name: pytorch
12
  pipeline_tag: video-classification
13
  ---
14
 
15
- # SurgRec — Surgical Video Pretraining Checkpoints
16
 
17
- Pretrained backbones for
18
- [**Scaling Video Pretraining for Surgical Foundation Models**](https://arxiv.org/abs/2603.29966)
19
- (arXiv:2603.29966). Code, splits, and the launch scripts used for the reported
20
- runs live in
21
- [Siiichenggg/SurgRec-open-source](https://github.com/Siiichenggg/SurgRec-open-source).
22
 
23
- **Authors.** Sicheng Lu, Zikai Xiao, Jianhui Wei, Danyu Sun, Qi Lu, Keli Hu,
24
- Yang Feng, Jian Wu, Zongxin Yang, Zuozhu Liu.
25
 
26
- These checkpoints are the product of that joint work. This repository is hosted
27
- under an individual account for distribution only; it does not indicate sole
28
- authorship.
29
 
30
- These are **pretrained backbones**, not task models. They carry no
31
- classification head and are meant to be fine-tuned on a downstream surgical
32
- video dataset.
33
 
34
- ![Overview of the SurgRec pretraining and evaluation framework](overview.jpg)
 
 
 
 
 
 
35
 
36
- *Overview of the proposed surgical video pretraining and evaluation framework.*
 
37
 
38
  ## Highlights
39
 
@@ -71,43 +72,41 @@ Top-1 accuracy (%) on the 16-dataset downstream benchmark, from Table 2 of the
71
  | SAR-RARP50 | 23.01 | 26.88 | 25.97 | 24.37 | 22.10 | **37.59** | 36.67 |
72
  | **Mean** | 32.56 | 33.87 | 28.25 | 35.92 | 34.02 | **45.28** | 39.11 |
73
 
74
- ## Files
75
 
76
- | File | Model | Architecture | Params | Objective |
77
- | --- | --- | --- | --- | --- |
78
- | `surgrec_mae.pth` | SurgRec-MAE | ViT-B/16, 12 blocks, width 768 | 94.2 M | Masked video reconstruction (VideoMAE-style) |
79
- | `surgrec_jepa.pt` | SurgRec-JEPA | ViT-L/16, 24 blocks, width 1024 | 303.9 M | Latent predictive pretraining (V-JEPA-style) |
80
-
81
- SHA256, for verifying downloads:
82
-
83
- ```text
84
- ce2c0298292d636f95c77a12c26fcdeb93de8b4a78e515a68d247de5e61759c5 surgrec_mae.pth
85
- 84040aa2e2fc015d03165903e6eec06282507f1d275e76efa97292457cd1bd3e surgrec_jepa.pt
86
- ```
87
 
88
- ### `surgrec_mae.pth`
 
89
 
90
- Recorded training configuration, read from the checkpoint itself:
91
 
92
  - `model = pretrain_videomae_base_patch16_224`
93
  - `mask_type = tube`, `mask_ratio = 0.9`, `decoder_depth = 4`
94
  - `input_size = 224`, `num_frames = 16`, `sampling_rate = 4`
95
- - `lr = 3e-4`, `batch_size = 64`
96
- - checkpoint taken at `epoch = 149`
97
 
98
- The archive contains `model`, `optimizer`, `scaler`, `epoch`, and `args`. The
99
- `model` entry holds both the encoder and the MAE decoder; fine-tuning loads the
100
- encoder and discards `decoder.*`. Load it with `model_key = model`.
101
 
102
- ### `surgrec_jepa.pt`
 
 
 
 
103
 
104
- - `encoder`, `predictor`, and `target_encoder`, each ViT-L/16
105
- - checkpoint taken at `epoch = 131`, `loss = 0.4225`
106
- - trained with `batch_size = 24` across `world_size = 8`
107
 
108
- Load the `encoder` entry for fine-tuning (`model_key = encoder`); weights are
109
- stored under a `module.backbone.` prefix, which the loader in the code
110
- repository strips.
 
 
 
111
 
112
  ## Usage
113
 
@@ -115,7 +114,6 @@ repository strips.
115
  git clone https://github.com/Siiichenggg/SurgRec-open-source.git
116
  cd SurgRec-open-source
117
  pip install -r requirements.txt
118
-
119
  hf download SichengLu/SurgRec --local-dir checkpoints
120
  ```
121
 
@@ -133,21 +131,15 @@ CKPT_ROOT=checkpoints DATA_ROOT=/path/to/split GPUS=8 \
133
  bash scripts/paper/finetune/jepa_v3_finetune_8gpu.sh
134
  ```
135
 
136
- ## Provenance
137
-
138
- `surgrec_mae.pth` is released from `videomae_e149.pth` and `surgrec_jepa.pt`
139
- from `jepa_v3_60w_100e_20w_130e.pt`; the names follow `MODEL_ZOO.md` in the code
140
- repository. See the paper for the pretraining corpus and its sampling
141
- procedure.
142
-
143
- ## License
144
 
145
- [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) — non-commercial
146
- use with attribution. The training code derives in part from VideoMAE, which
147
- carries the same obligation. See `LICENSE` and `NOTICE` in the code repository.
148
 
149
- These weights are a research artifact, not a medical device, and must not be
150
- used for clinical decisions.
 
151
 
152
  ## Citation
153
 
 
12
  pipeline_tag: video-classification
13
  ---
14
 
15
+ # SurgRec — Surgical Video Foundation Models
16
 
17
+ [![arXiv](https://img.shields.io/badge/arXiv-2603.29966-b31b1b.svg)](https://arxiv.org/abs/2603.29966)
18
+ [![GitHub](https://img.shields.io/badge/GitHub-SurgRec--open--source-181717.svg?logo=github)](https://github.com/Siiichenggg/SurgRec-open-source)
19
+ [![License: CC BY-NC 4.0](https://img.shields.io/badge/License-CC%20BY--NC%204.0-lightgrey.svg)](https://creativecommons.org/licenses/by-nc/4.0/)
 
 
20
 
21
+ Pretrained backbones from
22
+ [**Scaling Video Pretraining for Surgical Foundation Models**](https://arxiv.org/abs/2603.29966).
23
 
24
+ ![Overview of the SurgRec pretraining and evaluation framework](overview.jpg)
 
 
25
 
26
+ <p align="center"><em>Overview of the proposed surgical video pretraining and evaluation framework.</em></p>
 
 
27
 
28
+ ```bash
29
+ hf download SichengLu/SurgRec --local-dir checkpoints
30
+ ```
31
+
32
+ > These are **headless backbones** — no classification head. Fine-tune them on a
33
+ > downstream surgical video dataset with the
34
+ > [code repository](https://github.com/Siiichenggg/SurgRec-open-source).
35
 
36
+ **Sicheng Lu, Zikai Xiao, Jianhui Wei, Danyu Sun, Qi Lu, Keli Hu, Yang Feng,
37
+ Jian Wu, Zongxin Yang, Zuozhu Liu**
38
 
39
  ## Highlights
40
 
 
72
  | SAR-RARP50 | 23.01 | 26.88 | 25.97 | 24.37 | 22.10 | **37.59** | 36.67 |
73
  | **Mean** | 32.56 | 33.87 | 28.25 | 35.92 | 34.02 | **45.28** | 39.11 |
74
 
75
+ ## Models
76
 
77
+ | File | Model | Architecture | Params | Pretraining objective | Load with |
78
+ | --- | --- | --- | --- | --- | --- |
79
+ | `surgrec_mae.pth` | SurgRec-MAE | ViT-B/16 | 94.2 M | Masked video reconstruction (VideoMAE-style) | `model_key=model` |
80
+ | `surgrec_jepa.pt` | SurgRec-JEPA | ViT-L/16 | 303.9 M | Latent predictive (V-JEPA-style) | `model_key=encoder` |
 
 
 
 
 
 
 
81
 
82
+ <details>
83
+ <summary><b>Checkpoint internals, training configuration, and checksums</b></summary>
84
 
85
+ **`surgrec_mae.pth`** — configuration recorded inside the checkpoint:
86
 
87
  - `model = pretrain_videomae_base_patch16_224`
88
  - `mask_type = tube`, `mask_ratio = 0.9`, `decoder_depth = 4`
89
  - `input_size = 224`, `num_frames = 16`, `sampling_rate = 4`
90
+ - `lr = 3e-4`, `batch_size = 64`, taken at `epoch = 149`
 
91
 
92
+ The archive holds `model`, `optimizer`, `scaler`, `epoch`, and `args`. The
93
+ `model` entry contains both the encoder and the MAE decoder; fine-tuning loads
94
+ the encoder and discards `decoder.*`.
95
 
96
+ **`surgrec_jepa.pt`** — holds `encoder`, `predictor`, and `target_encoder`, each
97
+ ViT-L/16; taken at `epoch = 131`, `loss = 0.4225`, trained with
98
+ `batch_size = 24` across `world_size = 8`. Load the `encoder` entry; its weights
99
+ sit under a `module.backbone.` prefix, which the loader in the code repository
100
+ strips.
101
 
102
+ SHA256, for verifying downloads:
 
 
103
 
104
+ ```text
105
+ ce2c0298292d636f95c77a12c26fcdeb93de8b4a78e515a68d247de5e61759c5 surgrec_mae.pth
106
+ 84040aa2e2fc015d03165903e6eec06282507f1d275e76efa97292457cd1bd3e surgrec_jepa.pt
107
+ ```
108
+
109
+ </details>
110
 
111
  ## Usage
112
 
 
114
  git clone https://github.com/Siiichenggg/SurgRec-open-source.git
115
  cd SurgRec-open-source
116
  pip install -r requirements.txt
 
117
  hf download SichengLu/SurgRec --local-dir checkpoints
118
  ```
119
 
 
131
  bash scripts/paper/finetune/jepa_v3_finetune_8gpu.sh
132
  ```
133
 
134
+ ## License and intended use
 
 
 
 
 
 
 
135
 
136
+ Released for **non-commercial research**. These weights are a research artifact,
137
+ not a medical device, and must not be used for clinical diagnosis or treatment
138
+ decisions.
139
 
140
+ [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) — attribution
141
+ required, no commercial use. The code derives in part from VideoMAE, which
142
+ carries the same obligation; see `LICENSE` and `NOTICE` in the code repository.
143
 
144
  ## Citation
145