--- language: en license: apache-2.0 library_name: timm pipeline_tag: image-feature-extraction base_model: medarc/OpenMidnight-1.1 base_model_relation: finetune tags: - pathology - histopathology - feature-extraction - spectra --- # OpenMidnight-1.1-SPECTRA [MedARC](https://www.medarc.ai/) · [Sophont](https://sophont.med/) · [Paper (coming soon)](#paper) OpenMidnight-1.1-SPECTRA is [OpenMidnight-1.1](https://huggingface.co/medarc/OpenMidnight-1.1) adapted with **SPECTRA** by MedARC and Sophont. We trained on registered PLISM image tiles, aligning representations of the same tissue across scanners and stains with a contrastive objective. The learned updates are merged into the full model weights. The model maps RGB histology tiles to **1,536-dimensional embeddings** for downstream analysis. All inference weights are included; no separate base checkpoint, adapter, or cluster access is required. ## Inference In a Python environment: ```bash uv pip install torch torchvision "timm==1.0.30" huggingface_hub safetensors pillow ``` ```python import torch import timm from PIL import Image from timm.data import create_transform, resolve_model_data_config device = "cuda" # Use "cpu" to run without a GPU. model = timm.create_model( "hf-hub:medarc/OpenMidnight-1.1-SPECTRA", pretrained=True, ).eval().to(device) transform = create_transform(**resolve_model_data_config(model), is_training=False) image = Image.open("tile.png").convert("RGB") batch = transform(image).unsqueeze(0).to(device) with torch.inference_mode(): tokens = model.forward_features(batch) embedding = tokens[:, 0] print(embedding.shape) # torch.Size([1, 1536]) ``` The configuration uses `crop_pct=1.0` and `crop_mode="squash"` to resize the full image directly to 224 × 224 pixels. It uses bicubic interpolation and ImageNet normalization stored in `config.json`. The embedding uses the CLS token. These are tile embeddings. Slide-level predictions require a downstream model. ### Segmentation preprocessing The transform resizes the entire image to 224 × 224 without removing tissue. For square tiles, this preserves the aspect ratio; rectangular images are stretched to fit. The final center-crop operation generated by timm is a no-op because the image is already 224 × 224. The image and segmentation mask must cover the same tissue region. With a full-tile resize, decoder logits can be interpolated back to the original mask size before computing the loss or metrics, as THUNDER does. If resizing a discrete label mask instead, use nearest-neighbor interpolation. Exclude the CLS and register tokens when passing patch features to a segmentation decoder: `patch_tokens = tokens[:, model.num_prefix_tokens:]`. ## Paper Coming soon. We will add the paper link and citation here.