Satella-RAG-27B

Satella-RAG-27B is a 27B retrieval-grounded reasoning model fine-tuned from Qwen3.5-27B.

The model is designed for Retrieval-Augmented Generation (RAG): it is trained to prioritize supplied evidence, preserve citations when available, abstain when the evidence is insufficient, and remain useful for ordinary reasoning when no retrieved context is supplied.

Model repository: EphAsad/Satella-RAG-27B

Important: Satella-RAG-27B was fine-tuned and evaluated on text only. Qwen3.5-27B is a multimodal architecture, and the merged release retains that architecture, but Satella's visual behavior was not trained or benchmarked in this project. The results below should therefore be read as text/RAG results only.

What changed from Qwen3.5-27B?

Satella-RAG-27B uses supervised fine-tuning to emphasize five behaviors:

1. Evidence-first answering — retrieved evidence is treated as the primary source of truth.
2. Grounded abstention — when the supplied evidence does not support an answer, the model is trained to say so rather than invent one.
3. Citation preservation — document identifiers and inline citations are retained when present in the evidence format.
4. Resistance to irrelevant context — retrieved material that does not answer the question should not be treated as evidence.
5. Normal reasoning without retrieval — when no evidence is supplied, Satella can answer normally using the capabilities inherited from the base model.

This is an SFT specialization, not a new pre-trained foundation model.

Release contents

This repository contains:

• merged 16-bit/BF16 Transformers weights;
• the original LoRA adapter under adapter/;
• tokenizer / processor configuration;
• the canonical Satella system prompt;
• training and evaluation metadata under artifacts/;
• validated language-model GGUF exports:
  • Q8_0
  • Q6_K
  • Q5_K_M

A Q4_K_M artifact is not part of this release. Its export was deliberately stopped and it should not be assumed to exist.

The GGUF language-model files were smoke-tested with llama.cpp before upload. Multimodal projector files produced during conversion are auxiliary artifacts and were not independently evaluated for visual performance.

Training

Satella-RAG-27B was trained with 16-bit/BF16 LoRA, not 4-bit QLoRA.

Setting	Value
Base model	Qwen/Qwen3.5-27B
Training examples	22,804
Rendered training tokens	27,648,562
Max training sequence length	4,096 tokens
Max assistant answer length	2,048 tokens
Epochs	1
Learning rate	2e-5
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.0
Optimizer	AdamW 8-bit
Per-device batch size	4
Gradient accumulation	8
Effective batch size	32
Gradient checkpointing	Unsloth
Loss	response-only SFT
Trainable parameters	88,342,528
Trainable fraction	0.3219%
Final training loss	0.4689

LoRA adapters were attached to:

q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, and out_proj.

The vision tower was explicitly excluded from training.

Training mixture

The mixture was balanced by rendered token volume, not raw example count.

Source	Rows	Tokens	Token share
glaiveai/RAG-v1	13,803	15,197,161	54.97%
neural-bridge/rag-dataset-12000	5,047	5,526,856	19.99%
OpenDataArena/Spark-234K	3,124	6,080,714	21.99%
neural-bridge/rag-hallucination-dataset-1000	766	829,063	3.00%
Satella identity examples	64	14,768	0.05%

Held-out splits were created before training sampling. Exact prompt/context duplicates were removed across evaluation and training splits.

Examples exceeding the configured limits were rejected rather than silently truncated. In particular, Spark examples with assistant responses longer than 2,048 tokens were excluded.

The training datasets are not redistributed as part of this model release. Users should review the licenses and terms of the upstream datasets independently.

Evaluation

The base model and Satella-RAG-27B were evaluated on the same 90-example held-out RAG benchmark:

• 30 examples from Glaive RAG-v1;
• 30 examples from Neural Bridge RAG-12000;
• 30 examples from Neural Bridge RAG Hallucination.

These examples were held out before training-data sampling and decontaminated against the training split.

Metric	Qwen3.5-27B	Satella-RAG-27B	Delta
Overall reference F1	0.3185	0.7880	+0.4695
Glaive answer F1	0.4490	0.5887	+0.1396
Glaive citation F1	0.1667	0.9711	+0.8044
Neural RAG answer F1	0.4860	0.7752	+0.2892
Hallucination-set answer F1	0.0204	1.0000	+0.9796
Unanswerable / abstention score	0.0667	1.0000	+0.9333

How to interpret these results

These are project-specific held-out metrics, not broad claims that Satella-RAG-27B is generally superior to Qwen3.5-27B.

• Answer F1 is lexical word-overlap F1 against the held-out reference answer.
• Citation F1 compares extracted document citation IDs with the reference citation set.
• Abstention score measures whether the model correctly produces an insufficiency/abstention response on the held-out unanswerable set.
• The benchmark is small and intentionally focused on the behavior this fine-tune targets.
• Spark reasoning data was used during training but is not represented in this 90-example RAG benchmark, so these results should not be interpreted as a general reasoning benchmark.

Raw benchmark outputs and training metadata are included in the repository artifacts where available.

Canonical Satella system prompt

You are Satella, a retrieval-grounded reasoning model.

Your primary purpose is to reason over supplied evidence and produce accurate, concise, traceable answers.

When retrieved evidence is supplied:
- Treat the supplied evidence as the primary source of truth.
- Use only claims that are supported by the evidence unless the requested response mode explicitly permits relevant prior knowledge.
- Ignore irrelevant or distracting retrieved material.
- Cite supplied documents when citation identifiers are available.
- Never fabricate a citation or claim that evidence says something it does not.
- If the supplied evidence is insufficient to answer the question, state this clearly rather than inventing an answer.
- If supplied documents conflict, identify the conflict rather than silently choosing or inventing a resolution.

When no retrieved evidence is supplied:
- Answer normally using your reasoning and existing knowledge.
- Prefer clear, direct and concise responses.

You are Satella.

Recommended RAG prompt format

For grounded-only RAG:

### Retrieved evidence

Document:0
Text: <retrieved text>

Document:1
Text: <retrieved text>

### Response mode

GROUNDED

Answer only from the retrieved evidence.
Do not introduce unsupported outside information.
Preserve useful document citations.

### Question

<question>

For mixed evidence + prior knowledge, use MIXED and explicitly state that relevant prior knowledge is permitted.

Transformers usage

Qwen3.5 is a multimodal Transformers architecture. For text-only Satella use, typed text content blocks are the safest interface:

import torch
from transformers import AutoProcessor, AutoModelForMultimodalLM

model_id = "EphAsad/Satella-RAG-27B"

processor = AutoProcessor.from_pretrained(model_id)

model = AutoModelForMultimodalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": [
            {
                "type": "text",
                "text": "You are Satella, a retrieval-grounded reasoning model. "
                        "Treat supplied evidence as the primary source of truth. "
                        "If the evidence is insufficient, say so."
            }
        ],
    },
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": """### Retrieved evidence

Document:0
Text: Sample X must be incubated at 37 degrees Celsius.

### Response mode

GROUNDED

Answer only from the retrieved evidence.

### Question

At what temperature should Sample X be incubated?"""
            }
        ],
    },
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=False,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
    )

generated = output[0][inputs["input_ids"].shape[-1]:]
print(processor.decode(generated, skip_special_tokens=True))

For large-model inference, use an execution backend and precision appropriate for your available hardware.

GGUF / llama.cpp

Three language-model GGUF quantizations are included:

Quantization	Approximate uploaded size
Q8_0	27.05 GiB
Q6_K	20.89 GiB
Q5_K_M	18.19 GiB

Example text-only invocation:

./llama-cli \
  -m Satella-RAG-27B-Q5_K_M.gguf \
  -c 4096 \
  -n 512 \
  -p "You are Satella, a retrieval-grounded reasoning model..."

The base architecture is multimodal and conversion may also produce mmproj artifacts, but Satella's visual behavior was not fine-tuned or evaluated. Use those files experimentally rather than treating them as a validated visual release.

Intended use

Satella-RAG-27B is intended for:

• RAG assistants;
• evidence-grounded question answering;
• document and knowledge-base assistants;
• citation-aware generation;
• systems that need explicit refusal/abstention when retrieved evidence is insufficient;
• experimentation with retrieval-grounded generation.

It is designed to be the generation component of a RAG system. It does not perform retrieval by itself.

Limitations

• Fine-tuning and evaluation were text-only.
• Retrieval quality remains a major determinant of answer quality.
• Learned citation behavior is not a formal verification system; citations should still be checked against the source documents.
• The 90-example evaluation is deliberately narrow and should not be treated as a comprehensive capability benchmark.
• The model can still hallucinate, misunderstand evidence, or follow misleading retrieved text.
• The model was trained at a maximum sequence length of 4,096 tokens. Behavior beyond the fine-tuning context length was not evaluated as part of this release.
• No domain-specific medical, legal, financial, or safety-critical validation was performed.
• The visual capabilities inherited from Qwen3.5-27B were not validated after Satella fine-tuning.

For high-stakes use, retrieved sources, deterministic controls, and qualified human review should remain authoritative.

Reproducibility artifacts

The release includes lightweight artifacts describing the run, including:

• training configuration;
• dataset mixture summary;
• LoRA target-module audit;
• trainer metrics;
• base-model benchmark results;
• Satella benchmark results;
• benchmark comparison data;
• environment metadata.

The public training dataset itself is intentionally not bundled with the model.

License and upstream attribution

Satella-RAG-27B is derived from Qwen/Qwen3.5-27B, whose Hugging Face release is licensed under Apache-2.0.

This model repository uses the Apache-2.0 license. Upstream datasets have their own licenses and terms; using or reproducing the training mixture requires reviewing those sources independently.

Citation

If you use Satella-RAG-27B in a project, you can cite the model repository as:

@misc{satella-rag-27b,
  title        = {Satella-RAG-27B},
  author       = {EphAsad},
  howpublished = {Hugging Face model repository},
  year         = {2026}
}