AsteroRank-Lepton

We release AsteroRank-Lepton, a 4B SOTA open-weights reranker for STEM and general search.

The AsteroRank blog post shows more information about AsteroRank model family which includes Lepton, Drachma which is better, and Talent which outperforms every reranker known to date. The blog post presents full evaluation of 3 models, the idea of context reranking, and how to prompt search agents to write good context.

Model details

Property	Value
Developer	Asteromorph
Base model	Qwen/Qwen3.5-4B
Input	Text only: query, optional context, document
Output	Relevance score in (0, 1); higher is more relevant
License	CC BY-NC 4.0

Quickstart

We have zero day support with SGLang, vLLM and Sentence Transformers for using AsteroRank-Lepton.

SGLang

We keep base model Qwen3.5's standard configuration, so SGLang serves it without code changes. Below is the setup we tested, on one NVIDIA B200 (Blackwell) GPU with the lmsysorg/sglang:v0.5.20-cu130-runtime image:

PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True sglang serve \
  --model-path Asteromorph/AsteroRank-Lepton \
  --port 30000 \
  --attention-backend trtllm_mha \
  --mem-fraction-static 0.6 \
  --max-running-requests 64 \
  --max-mamba-cache-size 1280 \
  --cuda-graph-max-bs-decode 64 \
  --chunked-prefill-size 65536 \
  --max-prefill-tokens 65536 \
  --disable-prefill-cuda-graph

Known issues with SGLang 0.5.20:

Illegal memory access with large prefill chunks. Qwen3.5's gated RMSNorm kernel computes a 32-bit offset that overflows past 174,763 tokens, so keep --chunked-prefill-size below 174,764. --disable-prefill-cuda-graph stops 0.5.20 from capturing CUDA graphs up to the chunk size.
Mixed batches crash the scheduler. A /v1/rerank request batched together with a request that scores no tokens, such as a chat or /generate call, crashes the scheduler (sglang#34719, still open in 0.5.22; sglang#35052 proposes a fix). Send only rerank traffic to the server, and probe /health, not /health_generate.

Send the query, the context in instruct, and the candidate documents to SGLang's /v1/rerank:

curl http://localhost:30000/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "query": "para-selective borylation aryl sulfonate ion pair",
    "instruct": "The article discusses para-selective iridium-catalyzed C–H borylation of aryl sulfonate salts using ion-pairing with bulky ammonium cations to shield ortho and meta positions, enabling high para selectivity.",
    "documents": [
      "A bulky bipyridine ligand on iridium steers C–H borylation of monosubstituted arenes to the para position through steric repulsion alone.",
      "Ion pairing between aryl sulfonate anions and tetrabutylammonium cations blocks the ortho and meta sites, so iridium-catalyzed C–H borylation proceeds with high para selectivity.",
      "Suzuki–Miyaura coupling of aryl boronic esters with aryl halides under palladium catalysis."
    ],
    "return_documents": false
  }'

The response lists the documents from most to least relevant:

[
  {"score": 0.6792, "index": 1},
  {"score": 0.2814, "index": 0},
  {"score": 0.0474, "index": 2}
]

top_n limits how many documents come back, and return_documents: true echoes each document's text. Leave instruct out to rank on the query alone.

vLLM

vLLM serves the model through its pooling runner, as a one-label classifier. Below is the setup we tested, on one NVIDIA B200 (Blackwell) GPU with the vllm/vllm-openai:v0.32.0 image:

vllm serve Asteromorph/AsteroRank-Lepton \
  --runner pooling \
  --convert classify \
  --hf-overrides '{"architectures": ["Qwen3_5ForCausalLM"], "text_config": {"method": "from_2_way_softmax", "classifier_from_token": ["no", "yes"]}}' \
  --chat-template "$(hf download Asteromorph/AsteroRank-Lepton chat_template.jinja --quiet)" \
  --max-num-batched-tokens 65536

Keep --chat-template. Without it, vLLM scores the query and document joined as plain text, not in the prompt the model was trained on.
Keep method and classifier_from_token inside text_config. At the top level the classifier head is never built, and every score comes back 0.5.

Send the context as instruction to vLLM's /rerank:

curl http://localhost:8000/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "query": "para-selective borylation aryl sulfonate ion pair",
    "instruction": "The article discusses para-selective iridium-catalyzed C–H borylation of aryl sulfonate salts using ion-pairing with bulky ammonium cations to shield ortho and meta positions, enabling high para selectivity.",
    "documents": [
      "A bulky bipyridine ligand on iridium steers C–H borylation of monosubstituted arenes to the para position through steric repulsion alone.",
      "Ion pairing between aryl sulfonate anions and tetrabutylammonium cations blocks the ortho and meta sites, so iridium-catalyzed C–H borylation proceeds with high para selectivity.",
      "Suzuki–Miyaura coupling of aryl boronic esters with aryl halides under palladium catalysis."
    ]
  }'

The response lists the documents from most to least relevant:

{
  "results": [
    {"index": 1, "relevance_score": 0.6866, "document": {"text": "Ion pairing between aryl sulfonate anions ..."}},
    {"index": 0, "relevance_score": 0.2837, "document": {"text": "A bulky bipyridine ligand on iridium ..."}},
    {"index": 2, "relevance_score": 0.0482, "document": {"text": "Suzuki–Miyaura coupling of aryl boronic esters ..."}}
  ]
}

top_n limits how many documents come back. Leave instruction out to rank on the query alone.

Sentence Transformers

The repository includes a Sentence Transformers CrossEncoder configuration. We tested sentence-transformers 6.2.0:

pip install -U "sentence-transformers[image]"   # uv project: uv add "sentence-transformers[image]"

The image extra installs Pillow, which Qwen3.5's processor loads even for text.

from sentence_transformers import CrossEncoder

model = CrossEncoder("Asteromorph/AsteroRank-Lepton")

query = "para-selective borylation aryl sulfonate ion pair"
context = (
    "The article discusses para-selective iridium-catalyzed C–H borylation of aryl sulfonate salts "
    "using ion-pairing with bulky ammonium cations to shield ortho and meta positions, enabling high "
    "para selectivity."
)
documents = [
    "A bulky bipyridine ligand on iridium steers C–H borylation of monosubstituted arenes to the para position through steric repulsion alone.",
    "Ion pairing between aryl sulfonate anions and tetrabutylammonium cations blocks the ortho and meta sites, so iridium-catalyzed C–H borylation proceeds with high para selectivity.",
    "Suzuki–Miyaura coupling of aryl boronic esters with aryl halides under palladium catalysis.",
]
print(model.rank(query, documents, prompt=context))

[{'corpus_id': 1, 'score': 0.6792}, {'corpus_id': 0, 'score': 0.2814}, {'corpus_id': 2, 'score': 0.0474}]

Pass the context as prompt; leave it out to rank on the query alone. Installing flash-linear-attention and causal-conv1d speeds up Qwen3.5's linear-attention layers; without them, transformers falls back to slower reference kernels.