Instructions to use microsoft/Aion-1.0-Instruct-2.3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use microsoft/Aion-1.0-Instruct-2.3B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="microsoft/Aion-1.0-Instruct-2.3B", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("microsoft/Aion-1.0-Instruct-2.3B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use microsoft/Aion-1.0-Instruct-2.3B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "microsoft/Aion-1.0-Instruct-2.3B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/Aion-1.0-Instruct-2.3B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/microsoft/Aion-1.0-Instruct-2.3B
- SGLang
How to use microsoft/Aion-1.0-Instruct-2.3B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "microsoft/Aion-1.0-Instruct-2.3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/Aion-1.0-Instruct-2.3B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "microsoft/Aion-1.0-Instruct-2.3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/Aion-1.0-Instruct-2.3B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use microsoft/Aion-1.0-Instruct-2.3B with Docker Model Runner:
docker model run hf.co/microsoft/Aion-1.0-Instruct-2.3B
Aion 1.0 Instruct
Model Description
Aion Instruct serves as the foundational Small Language Model (SLM) in the Windows ecosystem, specifically optimized for on-device AI, text and operating system use cases. This 2.3B parameters base model delivers enterprise-grade language understanding and generation capabilities directly at the edge, enabling responsive, private, and cost-effective AI-powered scenarios without requiring constant cloud connectivity.
A further optimized version of Aion Instruct, coupled with Microsoftโs content moderation model, will become the Windows OS SLM by the end of January 2027. This Hugging Face version is meant to be an introduction to whatโs coming and a way to maximize transparency with our users, as well as welcome their feedback. We are publishing the unquantized model first, to be followed by quantized versions.
Model Details
- Languages: English, German, Spanish, Portuguese, Italian, French, Chinese, Japanese
- License: MIT
- Library names: PyTorch, ONNX, GGUF
- Task: Text generation
- Tags: Summarization, rewriting, chat
- Authors: Applied Sciences Group @ Microsoft Windows & Devices
- Related GitHub repository: microsoft/Aion-Instruct-Preview-Sample
Uses
The model was primarily trained for general-purpose text scenarios including document comprehension, content analysis, question answering, and natural language understanding. With context window extending to 32K tokens, it can process multi-page documents, lengthy email threads, and complex technical specifications entirely on-device, ensuring user data remains private.
Out-of-Scope Use
The model is not specifically designed or evaluated for every possible downstream purpose. Anyone accessing the model should bear in mind the guidance below.
- Developers should consider common limitations of language models as they select use cases, and evaluate and mitigate for accuracy, safety, and fairness before using within a specific downstream use case, particularly for high-risk scenarios.
- Developers should be aware of and adhere to applicable laws or regulations (including privacy, trade compliance laws, etc.) that are relevant to their use case.
- This model should not be used for:
- Generating harmful, violent, explicit, or offensive content.
- Creating misleading content, such as fake news or misinformation.
- Automated decision-making in critical contexts like medical diagnoses or legal judgments.
- Nothing contained in this Model Card should be interpreted as or deemed a restriction or modification to the license the model is released under.
Bias, Risks, and Limitations
With 2.3 billion parameters, Aion Instruct is a rather small language model. While this means the model is taking up less disk space and can be quite fast, it also comes with trade-offs.
In comparison to large language models, whose size enables storing vast amounts of factual knowledge and grasping subtle context, logic, and nuance, small language models such as Aion Instruct can miss fine details, provide misstatements, or show inability to follow complex multi-step reasoning. Larger models also have larger context and the memory needed to remember facts accurately, as well as the reasoning to see through trick questions. Smaller models lack the storage and reasoning depth to consistently do both.
While Microsoftโs best practices were followed while training and safety alignment was conducted, anyone using the model should bear in mind that Aion Instruct does not have the complex reasoning needed to understand all the nuances of safety, including attempts to induce unsafe behavior. Aion Instruct may be susceptible to adversarial attacks.
XPIA, DIC, and ECI are amongst the key metrics to improve in future releases:
Scores are direction-normalized so higher is better (0โ100).
While we continue to work towards closing performance gaps across languages and supporting more languages, output quality may differ depending on the language.
We also remain committed to improving the groundedness and safety of Aion Instruct. One way of limiting harmful output generation are content moderation models โ a local model from that group will be used alongside Aion Instruct when the model becomes available in Windows (General Availability coming by end of Jan 2027).
Recommendations
The model was not trained to excel at world knowledge. Users should take note of recommended uses (see section: Uses), use their best judgement, and follow laws and regulations when interacting with the model.
Your feedback is important for helping us improve Aion Instruct. If you encounter harmful, inaccurate, or otherwise unexpected outputs from the model, please provide feedback.
Training Data
Our training data is an extension of the data used for training Phi Silica (the previous generation Windows OS SLM) and includes a wide variety of sources from:
- Publicly available documents filtered rigorously for quality;
- Newly created synthetic data for the purpose of teaching math, coding, common sense reasoning, general knowledge of the world (science, daily activities, theory of mind, etc.).
Multilingual data constitutes about 6% of our overall data. We are focusing on the quality of data that could potentially improve the reasoning ability for the model, and we filter the publicly available documents to contain the correct level of knowledge.
Evaluation
All benchmarks were run with EleutherAIโs lm-evaluation-harness where a task implementation was available. Otherwise, the official implementation from each benchmarkโs own repository was used. For benchmarks that need an LLM judge (marked โ ), GPT-4o was the judge model. Scores are percentages unless stated otherwise.
| Category | Benchmark | Score | Notes |
|---|---|---|---|
| Knowledge | MMLU | 55.15 | |
| Knowledge | MMLU-Pro | 33.35 | |
| Knowledge | MMMLU | 45.51 | Multilingual MMLU |
| Knowledge | GPQA | 28.18 | |
| Math | GSM8K | 74.91 | Loose matching |
| Math | GSMPlus | 49.56 | |
| Math | MATH-500 | 48.00 | |
| Instruction following | IFEval | 83.90 | Mean of prompt-level strict and instruction-level strict |
| Instruction following | IFBench | 32.00 | Prompt-level loose |
| Instruction following | Multi-IF | 64.70 | |
| Translation | chrF++ | 43.18 |
Model Overview
| Type | Causal language model (instruction-tuned) |
| Parameters | ~2.29B (tied input/output embeddings) |
| Layers | 28 |
| Hidden size | 2048 |
| MLP intermediate size | 8192 (SiLU) |
| Attention | GQA: 24 query heads, 8 key/value heads, head dim 128 |
| Attention pattern | Hybrid: 21 sliding-window layers (window 2048) + 7 full-attention layers |
| Positional encoding | RoPE (ฮธ = 32000); YaRN scaling (factor 16) on full-attention layers |
| Context length | 32,768 tokens |
| Vocabulary size | 200,029 |
| Precision | bfloat16 |
| Knowledge cutoff | 2025-12 |
Quickstart
The model uses custom modeling code, so trust_remote_code=True is required. It was exported with transformers 5.x.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "asgmodels/Aion-Instruct-v1"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
dtype="auto",
device_map="auto",
trust_remote_code=True,
)
messages = [
{"role": "user", "content": "Give me a short introduction to large language models."},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Chat Format
Conversations are formatted with the included chat template:
<|system|>You are Aion Instruct, a large language model trained by Microsoft.
Knowledge cutoff: 2025-12
Current date: YYYY-MM-DD<|end|><|user|>Hello!<|end|><|assistant|>Hi! How can I help?<|end|>
- Every turn ends with
<|end|>, which is also the EOS token. - A leading
systemmessage is appended to the default system preamble. - Optional template variables:
model_identity,knowledge_cutoff, andcurrent_date.
Tool Calling
Pass tools via apply_chat_template(..., tools=tools). The tool schemas are injected into the system prompt between <|tool|> and <|/tool|>, and the model emits calls as:
<|tool_call|>{"name": "<function-name>", "arguments": {...}}<|/tool_call|>
Tool results (role: "tool" messages) are returned to the model inside a user turn prefixed with <|tool_response|>.
Files
| File | Description |
|---|---|
model.safetensors |
Model weights (bfloat16) |
config.json |
Model configuration |
generation_config.json |
Default generation settings |
configuration_aion_instruct.py |
Custom configuration class (AionInstructConfig) |
modeling_aion_instruct.py |
Custom model implementation (AionInstructForCausalLM) |
tokenizer.json, tokenizer_config.json |
Tokenizer |
chat_template.jinja |
Chat template |
License
This model is released under the MIT License.
Model Card Contact
- Downloads last month
- -

docker model run hf.co/microsoft/Aion-1.0-Instruct-2.3B