---
license: mit
library_name: peft
base_model: Qwen/Qwen3-4B
tags:
  - astronomy
  - spectroscopy
  - multimodal
  - desi
  - lora
---

# AstroSpecLM

AstroSpecLM is a spectrum-grounded language model for DESI one-dimensional
spectra. This repository contains the paper-aligned rank-16 checkpoint.

## Model components

- `llm_lora/`: rank-16 LoRA adapter for `Qwen/Qwen3-4B`.
- `projector.pt`: mapping from SpecMAE tokens to Qwen hidden states.
- `spectral_token_embeddings.pt`: learned spectral boundary-token embeddings.
- `spec_encoder/model.safetensors`: frozen SpecMAE spectrum encoder.
- `tokenizer/`: tokenizer files with the three spectral special tokens.
- `model_config.yaml`: inference architecture and paths.

The Qwen3-4B base weights are not redistributed here. They are loaded from the
upstream `Qwen/Qwen3-4B` repository.

## Architecture

```text
DESI spectrum -> frozen SpecMAE encoder -> projector -> Qwen3-4B + LoRA -> text
```

The spectrum input is a preprocessed flux vector of length 7,781. The encoder
uses patch size 20 and stride 10. The projector produces 130 spectral tokens.

## Checkpoint

This is the best checkpoint from the one-epoch rank-16 SFT run:

- training step: 9,697
- validation loss: 0.8958333147
- LoRA rank / alpha: 16 / 32

Use this checkpoint with the AstroSpecLM source release and pass the downloaded
model directory as `--adapter-dir`. After downloading, replace
`/absolute/path/to/AstroSpecLM` in `model_config.yaml` with the absolute local
path of the downloaded model directory.

```bash
hf download sjh11/AstroSpecLM --local-dir /absolute/path/to/AstroSpecLM

python3 scripts/chat.py \
  --config /absolute/path/to/AstroSpecLM/model_config.yaml \
  --adapter-dir /absolute/path/to/AstroSpecLM \
  --spectrum-npy /path/to/demo_flux.npy
```

## Scope and limitations

This is a research checkpoint, not a general-purpose scientific measurement
pipeline. Its generated descriptions and inferred physical properties require
independent validation before scientific use. Training datasets and the Qwen
base model are governed by their respective licenses.