File size: 4,956 Bytes
2a10ae7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 | ---
license: apache-2.0
language:
- en
library_name: pytorch
base_model: Qwen/Qwen2.5-7B-Instruct
datasets:
- WinstonHu/LoC-Path-Data
tags:
- loc-path
- pathology
- whole-slide-imaging
- visual-question-answering
- lora
---
# LoC-Path
Pretrained LoC-Path weights for whole-slide pathology understanding from
pre-extracted CONCH-v1 features. This release uses a **128-latent resampler
(`L=128`)**, followed by routing to **96 visual latents (`M=96`)** for CARA.
The language backbone is **Qwen2.5-7B-Instruct**, with LoRA rank 128.
## Released checkpoints
| File | Training checkpoint | Purpose |
|---|---|---|
| `checkpoints/mae.pth` | 100,000 iterations | Dedicated 128-latent MAE pretraining weights, including the reconstruction decoder |
| `checkpoints/stage1.pth` | 3,615 iterations | Complete visual weights after report alignment |
| `checkpoints/stage2.pth` | 20,480 iterations | Complete visual weights and Qwen LoRA for inference or initialization |
Each file is a PyTorch dictionary containing a tensor-only `state_dict`.
Checkpoints are consolidated in FP16 using the repository's ZeRO loader. Frozen
visual modules omitted by historical ZeRO saves are carried forward from their
MAE/Stage-1 sources into the complete exported state. A Stage-2 inference run
requires only `stage2.pth`, its matching config, the Qwen base model and features.
Qwen base weights are downloaded from
[Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) by the
LoC-Path loader. CONCH weights are obtained separately when extracting new
features. This repository distributes the trained LoC-Path visual modules and
LoRA; original image/report assets and optimizer/scheduler states are absent.
The [model manifest](model_manifest.json) records source checkpoints, tensor
counts, sizes and SHA256 hashes. The [validation summary](validation.json)
records configuration/weight checks and execution tests.
## Download
Install the LoC-Path training package using its README, then run:
```bash
export LOCPATH_MODEL_ROOT=$PWD/pretrained_models/LoC-Path
uv run --locked python - <<'PY'
import os
from huggingface_hub import snapshot_download
snapshot_download(
repo_id="WinstonHu/LoC-Path",
repo_type="model",
revision="v1.0",
local_dir=os.environ["LOCPATH_MODEL_ROOT"],
)
PY
```
For inference-only use, optionally add
`allow_patterns=["checkpoints/stage2.pth", "configs/locpath_stage2.py", "README.md", "model_manifest.json", "validation.json", "LICENSE"]`
to the download call.
## Run inference
Download the prepared WSI-Bench/BCNB data from
[WinstonHu/LoC-Path-Data](https://huggingface.co/datasets/WinstonHu/LoC-Path-Data/tree/v1.0).
Run from the installed LoC-Path code directory:
```bash
export LOCPATH_DATA_ROOT=$PWD/data/LoC-Path-Data
export CONFIG="$LOCPATH_MODEL_ROOT/configs/locpath_stage2.py"
export CHECKPOINT="$LOCPATH_MODEL_ROOT/checkpoints/stage2.pth"
export FEATURE_ROOT="$LOCPATH_DATA_ROOT/features/conch_v1"
export DATA_DIR="$LOCPATH_DATA_ROOT/annotations/test/wsi_bench"
export NPROC_PER_NODE=1
CUDA_VISIBLE_DEVICES=0 uv run --locked bash scripts/test_vis_adapter.sh \
--image_feature_suffix .npy --router-top-m 96 \
--sample_num 61440 --max_new_tokens 512
```
For BCNB, set `DATA_DIR="$LOCPATH_DATA_ROOT/annotations/test/external"` and run
the same command. Its JSON points to the included numeric feature CSVs.
To use an already downloaded Qwen base, set
`LOCPATH_LLM_PATH=/path/to/Qwen2.5-7B-Instruct`.
Use the supplied configs: all three stages have 128 latent queries. Set `CONFIG`
to the corresponding file when invoking a training launcher:
| Launcher | Config | Initialization |
|---|---|---|
| `scripts/train_resampler_pretrain.sh` | `configs/locpath_mae.py` | Train MAE from scratch, or initialize using the MAE weights |
| `scripts/train_stage1.sh` | `configs/locpath_stage1.py` | `LOCPATH_RESAMPLER_CKPT` points to `checkpoints/mae.pth` |
| `scripts/train_stage2.sh` | `configs/locpath_stage2.py` | `LOCPATH_RESAMPLER_CKPT` points to MAE; `LOCPATH_STAGE2_PRETRAINED_PTH` points to Stage 1 |
Set feature and annotation environment variables as described in the training
package README. Training starts a new optimizer/scheduler run; these exports
are weights for initialization and inference.
## Validation and use
The three exports were loaded into their matching model architectures with all
required LoC-Path tensors present and exact loaded values. The dedicated
128-latent Stage-2 model was tested on an actual Qwen2.5-7B backbone using
prepared WSI-Bench and BCNB samples. Execution checks are distinct from a full
benchmark-metric reproduction. Use this model for computational pathology
research with the corresponding source-data and feature-extractor terms.
LoC-Path weights are released under Apache-2.0. Attribute the LoC-Path project
and the Qwen base model. Data and CONCH terms are documented in the prepared-data
repository and apply to those assets separately.
|