OmniSpectra: A Unified Foundation Model for Universal Spectra Representation Learning
Abstract
OmniSpectra is a wavelength-native spectral foundation model that uses continuous coordinates and physics-grounded decoders to learn cross-instrument representations without destructive resampling, achieving high performance with far fewer parameters.
Standard sequence models encode position by token rank rather than physical coordinate, typically forcing practitioners to destructively resample continuous physical signals such as spectra observed across heterogeneous instruments. We identify this coordinate mismatch as a structural limitation in cross-instrument representation learning, and propose OmniSpectra, a wavelength-native foundation model for spectral representation learning: a continuous-coordinate masked autoencoder that discards the integer pixel indexing of sequence modeling. Mapping physical coordinates to a logarithmic basis turns the multiplicative scaling induced by cosmological redshift into a uniform additive shift. In continuous rotary position embeddings, this shared shift cancels from relative token distances rather than producing grid-dependent positional distortion. We further counter the spectral bias inherent to implicit neural decoders with a physics-grounded, resolution-gated Fourier decoder that provides anti-aliasing. Pretrained on eight surveys and benchmarked against seven baselines, our model reaches state-of-the-art performance with only 12.2M parameters while outperforming substantially larger models, indicating that encoding physical structure can substitute for parameter scale.
Community
Hi everyone! Co-author here 馃憢
Spectra from different instruments sample the same wavelengths on very different pixel grids (up to 25脳 apart in our training corpus). Standard sequence models position tokens by pixel index, so the same line (e.g. H伪) gets a different positional code on every instrument. The usual fix is to resample everything onto a common grid, which blurs narrow lines.
OmniSpectra is a wavelength-native masked autoencoder that removes this mismatch:
- Position = log 位, not pixel index. Tokens are anchored to physical wavelength. In log space, cosmological redshift becomes one uniform shift, which cancels in continuous RoPE, so attention depends on physical wavelength ratios rather than the detector grid.
- Resolution-gated Fourier decoder. Reconstructs flux at each instrument's native wavelengths. A gate set by the instrument's line-spread function suppresses detail the spectrograph cannot actually resolve.
- One set of weights, no resampling. Pretrained on 6.4M spectra from 8 surveys (SDSS, BOSS, eBOSS, SEGUE-1/2, DESI, APOGEE, GALAH).
With only 12.2M parameters, it reaches state-of-the-art results across stellar parameters, galaxy properties, source classification and redshift, and outperforms much larger models such as AstroCLIP and AION-1. It is especially strong when labels are scarce. Pretrained on SDSS alone, it still transfers to APOGEE, whose near-infrared range it never saw during pretraining, with little degradation.
Happy to answer questions!
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper