KeyAI-LLM/Sage-A1.5-35B-A3B-8bit-MLX

KeyAILLM/Sage-A1.5-35B-A3B-4-8bit-MLX

This repository provides an 8-bit quantized MLX version of the original InternScience/Sage-A1.5 model, converted and optimized for Apple Silicon using mlx-vlm.

• Original Non-Quantized Model: InternScience/Sage-A1.5
• Quantization Tool: Converted using mlx_vlm with 8-bit quantization (-q 8).
• Hardware Requirement: Apple Silicon (Mac M-series) with unified memory.

---

Model Overview

Sage‑A1.5 is a 35B Mixture-of-Experts (MoE) agentic vision-language model based on Qwen3.6 and Agents-A1 from Shanghai AI Laboratory. It is specialized in long-horizon scientific problem solving, agentic orchestration, tool-use, and multi-step reasoning.

This 8-bit MLX release significantly reduces memory footprint and latency while maintaining strong reasoning and multimodal performance on Apple Silicon Macs.

---

Recommended Generation Parameters

For optimal output quality, stability, and reasoning performance with this 8-bit quant, the following sampling parameters are recommended:

Parameter	Value
temperature	0.8
top_p	0.95
top_k	40
min_p	0.05
presence_penalty	1.5
repetition_penalty	1.1

---

Installation

Ensure you have Python 3.10+ and install mlx-vlm:

pip install -U mlx-vlm

---

Usage

1. Command Line Interface (CLI)

Generate text responses using the recommended sampling parameters:

python -m mlx_vlm.generate \
  --model KeyAILLM/sage-a1.5-8bit-mlx \
  --prompt "Solve the following problem step by step: ..." \
  --max-tokens 2048 \
  --temp 0.8 \
  --top-p 0.95

For multimodal inputs (text + image):

python -m mlx_vlm.generate \
  --model KeyAILLM/sage-a1.5-8bit-mlx \
  --image "path/to/image.png" \
  --prompt "Analyze this diagram and explain the key findings." \
  --max-tokens 2048 \
  --temp 0.8 \
  --top-p 0.95

---

2. Python API

from mlx_vlm import load, generate

model_path = "KeyAILLM/sage-a1.5-8bit-mlx"

# Load the quantized model and processor
model, processor = load(model_path)

prompt = "Explain the difference between synchronous and asynchronous agentic verification."
formatted_prompt = processor.apply_chat_template(
    [{"role": "user", "content": prompt}],
    add_generation_prompt=True,
    tokenize=False
)

response = generate(
    model,
    processor,
    prompt=formatted_prompt,
    max_tokens=2048,
    temp=0.8,
    top_p=0.95,
    repetition_penalty=1.1,
    verbose=True
)
print(response)

---

3. Local OpenAI-Compatible Server

You can host a local API server using mlx_vlm.server:

python -m mlx_vlm.server \
  --model KeyAILLM/sage-a1.5-8bit-mlx \
  --port 8000

Connect using standard OpenAI client libraries with the recommended settings:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")

response = client.chat.completions.create(
    model="KeyAILLM/sage-a1.5-8bit-mlx",
    messages=[
        {"role": "system", "content": "You are a helpful scientific reasoning assistant."},
        {"role": "user", "content": "Hello! What are your core capabilities?"}
    ],
    temperature=0.8,
    top_p=0.95,
    presence_penalty=1.5,
    max_tokens=2048,
    extra_body={
        "top_k": 40,
        "min_p": 0.05,
        "repetition_penalty": 1.1
    }
)
print(response.choices[0].message.content)

---

Quantization Details

• Source model: InternScience/Sage-A1.5
• Quantization mode: 8-bit linear quantization
• Converted with:

  python -m mlx_vlm.convert \
    --hf-path InternScience/Sage-A1.5 \
    --mlx-path ./sage-a1.5-8bit-mlx \
    -q 8
  

---

Original Work & Citation

Please cite the original authors if you use this model:

@misc{lou2026sagea15longhorizonexecutionscientific,
      title={Sage-A1.5: From Long-Horizon Execution to Scientific Exploration with Capability Composition}, 
      author={Sage-A1.5 Team},
      year={2026},
      eprint={2610.28578},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2610.28578}, 
}