LANYUE-HiViT-Base

Six-modality LANYUE-HiViT-Base pretrained weights for lunar representation learning. The model extends the five-modality spherical-geolocation model with the LROC NAC branch through three epochs of distillation from the epoch-12 teacher.

Code · Benchmark

Released checkpoint

File	Description
lanyue-hivit-base.pth	Model tensors with embedded architecture configuration and preprocessing statistics.
model_config.json	Matching six-modality model configuration.
preprocessing.json	Saved pretraining normalization and signal-scaling rules.
SHA256SUMS	Download integrity checksums.

The PyTorch file supports torch.load(..., weights_only=True). It contains 678 model tensors, including the shared encoder and reconstruction decoders. Optimizer/scaler state, machine-specific paths and unused legacy modules are excluded. Retained tensors keep their original dtype and values.

Architecture and inputs

Use lanyue_nac with the model configuration stored in the checkpoint.

Model input key	Observation	Patch size
ce2_dom	CE-2 DOM	16
ce2_dem	CE-2 DEM	16
nasa_dom	LROC WAC	8
nasa_dem	NASA lunar elevation	8
minirf_sar	Mini-RF SAR	16
nac	LROC NAC	16

The shared encoder has embedding dimension 512, stage depths 2/2/20 and eight attention heads. Each reconstruction decoder has four blocks. Dense feature extraction accepts any available subset, including NAC alone. Arrays must be [B,1,H,W] with dimensions divisible by their patch size. Physical scales are recorded in model_config.json; adjust model.modality_physical_scales when the task's pixel scale differs.

Download and load

Weight access requires manual approval. Sign in to Hugging Face and request access with the account you will use for downloading. After approval, run hf auth login with a read token from that account before downloading the weights.

Clone the code and install its model dependencies:

git clone https://github.com/LLuKaoo/LANYUE.git
cd LANYUE/LANYUE
python -m pip install -r requirements.txt
python -m pip install huggingface_hub
hf auth login  # After your access request is approved

Run the following from that model directory:

import torch
from huggingface_hub import hf_hub_download
from lanyue.models import lanyue_nac
from lanyue.features import encode_features

path = hf_hub_download("LLukaoo/LANYUE", "lanyue-hivit-base.pth")
checkpoint = torch.load(path, map_location="cpu", weights_only=True)
model = lanyue_nac(**checkpoint["config"]["model"])
model.load_state_dict(checkpoint["model"], strict=True)
model.eval()

# Illustrative normalized inputs; apply the appropriate image preprocessing.
inputs = {
    "nasa_dom": torch.randn(1, 1, 48, 88),
    "nasa_dem": torch.randn(1, 1, 80, 144),
}
with torch.no_grad():
    tokens = encode_features(model, inputs)
    descriptor = torch.cat([x.mean(1) for x in tokens.values()], dim=-1)
print(descriptor.shape)  # [1, 1024]

Coordinates can be supplied as inputs["center_lonlat"], in longitude/latitude order and degrees. Missing coordinates use the learned missing-location vector. The feature API does not automatically normalize or resize observations. Use the saved pretraining statistics for matching source imagery, and the task configuration when reproducing downstream experiments. Task prediction heads are trained separately and are not bundled in this pretrained checkpoint.

License and citation

MIT. Source-code attribution is retained in NOTICE. Citation details will be added with the paper release.