Instructions to use OVHaiLLM/Olala-v1.0-summarise with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OVHaiLLM/Olala-v1.0-summarise with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="OVHaiLLM/Olala-v1.0-summarise", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("OVHaiLLM/Olala-v1.0-summarise", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use OVHaiLLM/Olala-v1.0-summarise with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "OVHaiLLM/Olala-v1.0-summarise" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OVHaiLLM/Olala-v1.0-summarise", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/OVHaiLLM/Olala-v1.0-summarise
- SGLang
How to use OVHaiLLM/Olala-v1.0-summarise with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "OVHaiLLM/Olala-v1.0-summarise" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OVHaiLLM/Olala-v1.0-summarise", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "OVHaiLLM/Olala-v1.0-summarise" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OVHaiLLM/Olala-v1.0-summarise", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use OVHaiLLM/Olala-v1.0-summarise with Docker Model Runner:
docker model run hf.co/OVHaiLLM/Olala-v1.0-summarise
Olala-v1.0-summarise
Olala-v1.0-summarise is a document summarization model, fine-tuned from Olala-v1.0-mini-7A1B. It reasons before it writes the summary.
- Many document types: news, dialogues, forum posts, earnings calls, bills, government reports and scientific papers.
- Structured and faithful: trained to write structured summaries (sections or labelled bullets) that stay faithful to the source, and to follow the requested form ("in one sentence", "as bullet points").
- Concise: its summaries are shorter than those of 16 of the 17 compared models.
Model details
| Model | Description |
|---|---|
| Olala-v1.0-mini-7A1B-base | Pre-trained base model (12.8T tokens) |
| Olala-v1.0-mini-7A1B-SFT | After supervised fine-tuning, before RL |
| Olala-v1.0-mini-7A1B | Final post-trained model: reasoning, chat and tool use |
| Olala-v1.0-summarise | Specialized for document summarization |
| Olala-v1.0-translate | Specialized for machine translation, including financial documents |
| Property | Specification |
|---|---|
| Checkpoint | Fine-tuned from Olala-v1.0-mini-7A1B for summarization |
| Parameters | 6.8B total, about 1B active per token |
| Architecture | 36 layers: 29 Mamba-3 and 7 attention layers, mixture-of-experts with 256 routed experts (6 active) |
| Context length | 32K tokens (input + output) |
| Languages | English (training documents are English except EUR-Lex) |
| License | Apache 2.0 |
Inference
Sampling parameters: temperature=0.8, top_p=0.9, top_k=40, reasoning_effort="medium". Cap
max_tokens: 99% of generations finish under 2K tokens, and 4K to 8K is a safe cap.
vLLM: always serve this model with
--mamba-ssm-cache-dtype float32. vLLM's default keeps the Mamba recurrent state in bf16, and long generations then degrade into repetition loops.
vLLM quick start (vLLM fork, installed as for Olala-v1.0-mini-7A1B):
vllm serve OVHaiLLM/Olala-v1.0-summarise \
--trust-remote-code \
--mamba-ssm-cache-dtype float32 \
--reasoning-parser olala \
--max-model-len 32688 --port 8000
curl -s http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
"model": "OVHaiLLM/Olala-v1.0-summarise",
"messages": [{"role": "user", "content": "Write an executive summary of the following government report: why the work was done, what was found, and what is recommended.\n\n<report text>"}],
"temperature": 0.8, "top_p": 0.9, "top_k": 40, "max_tokens": 8192,
"chat_template_kwargs": {"reasoning_effort": "medium"}
}'
The summary is in choices[0].message.content and the reasoning in reasoning_content.
Transformers quick start:
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "OVHaiLLM/Olala-v1.0-summarise"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True, dtype="bfloat16",
device_map="cuda") # or "cpu" without a GPU (slow)
document = open("article.txt").read()
messages = [{"role": "user", "content": "Summarize the following news article in one sentence.\n\n" + document}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, reasoning_effort="medium",
return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**inputs, max_new_tokens=4096, use_cache=True,
do_sample=True, temperature=0.8, top_p=0.9, top_k=40)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=False))
Pass use_cache=True (config.json sets use_cache: false) and generate one prompt at a time.
Prompting
Put the instruction first, then a blank line, then the document. State the form you want ("in one sentence", "as short bullet points", "in one or two paragraphs"). With an open instruction ("Summarize the following paper."), the model tends to write a structured summary with sections.
reasoning_effort can be disabled, low, medium (default) or high. It barely changes ROUGE; medium is the
setting all the numbers on this card use.
Chat template
The model uses the general Olala 1.0 mini chat template (chat_template.jinja, the same as
Olala-v1.0-mini-7A1B). It writes its reasoning in an analysis channel, then the
summary in a final channel:
<|im_start|>assistant<|channel_start|>analysis<|content|>...reasoning...<|channel_end|><|channel_start|>final<|content|>...summary...<|channel_end|><|im_end|>
Use tokenizer.apply_chat_template() with add_generation_prompt=True to format messages automatically, and set
the reasoning effort with reasoning_effort=... (in vLLM, only inside chat_template_kwargs:
"chat_template_kwargs": {"reasoning_effort": "medium"}, not the top-level reasoning_effort parameter).
With the vLLM reasoning parser, the summary is in content and the reasoning in reasoning_content.
Performance
Benchmarks
Local summarization suite over 11 public test sets: up to 1,000 documents each (seed 42), documents cut at 12K
words, one-turn prompt <instruction>\n\n<document>. ROUGE-Lsum F1 times 100 (stemmed), plain average over
tasks. Length ratio = summary words / reference words.
| Test set (ROUGE-Lsum) | Olala v1.0-summarise 7B-A1B |
Falcon H1-7B-Instruct 7B |
Falcon H1-3B-Instruct 3B |
Mistral 7B-Instruct-v0.3 7B |
NVIDIA Nemotron-3-Nano-4B 4B |
Granite 4.1-8B 8B |
Qwen3.5-4B 4B |
Gemma 4-E2B-it 2B effective |
LFM2-2.6B 2.6B |
Qwen3 4B-Instruct-2507 4B |
LFM2.5 1.2B-Instruct 1.2B |
Qwen3.5-2B 2B |
SmolLM3-3B 3B |
OLMoE 1B-7B-0924-Instruct 7B-A1B |
LFM2-8B-A1B 8B-A1B |
Gemma 3-4B-it 4B |
LFM2.5 8B-A1B 8B-A1B |
LFM3-1.7B 1.7B |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| News | ||||||||||||||||||
| CNN/DailyMail | 33.0 | 33.5 | 34.1 | 31.8 | 31.7 | 29.9 | 29.2 | 30.6 | 29.0 | 29.5 | 32.9 | 29.8 | 32.5 | 29.1 | 28.9 | 27.1 | 28.2 | 28.0 |
| XSum | 16.5 | 21.8 | 22.4 | 18.3 | 16.7 | 18.4 | 18.1 | 17.5 | 16.9 | 18.0 | 16.9 | 16.3 | 16.7 | 16.9 | 16.4 | 18.6 | 15.7 | 14.8 |
| Multi-News | 36.3 | 37.2 | 37.6 | 38.6 | 36.6 | 37.6 | 37.0 | 34.1 | 37.0 | 36.9 | 33.9 | 37.0 | 39.1 | 36.4 | 33.7 | 36.1 | 32.5 | 34.6 |
| Dialogue | ||||||||||||||||||
| SAMSum | 33.4 | 32.8 | 31.5 | 32.5 | 33.1 | 33.1 | 28.8 | 32.2 | 30.1 | 32.9 | 29.4 | 29.6 | 33.1 | 30.6 | 28.9 | 28.9 | 29.6 | 30.1 |
| DialogSum | 28.1 | 26.5 | 24.4 | 24.7 | 27.1 | 27.6 | 22.7 | 23.3 | 24.3 | 26.4 | 23.2 | 24.7 | 25.0 | 21.3 | 24.2 | 21.6 | 24.3 | 21.5 |
| Forum posts | ||||||||||||||||||
| TL;DR | 21.2 | 25.2 | 18.5 | 18.7 | 18.1 | 17.9 | 16.6 | 19.0 | 16.2 | 18.2 | 18.2 | 15.6 | 17.9 | 21.7 | 16.3 | 17.5 | 15.7 | 15.7 |
| Earnings calls | ||||||||||||||||||
| ECTSum | 15.9 | 15.4 | 15.0 | 14.5 | 13.7 | 12.4 | 15.8 | 13.2 | 13.1 | 11.5 | 13.4 | 13.6 | 13.2 | 9.4 | 12.1 | 13.1 | 12.5 | 11.3 |
| Government and legal | ||||||||||||||||||
| BillSum | 38.8 | 40.3 | 40.0 | 44.5 | 37.3 | 37.0 | 36.0 | 37.2 | 41.0 | 31.7 | 43.6 | 35.0 | 38.8 | 38.6 | 41.0 | 35.2 | 35.7 | 32.9 |
| GovReport | 43.0 | 49.4 | 48.0 | 49.6 | 47.4 | 51.1 | 50.9 | 49.6 | 46.5 | 49.1 | 44.2 | 49.9 | 51.4 | 45.8 | 42.7 | 45.7 | 41.4 | 44.3 |
| Science | ||||||||||||||||||
| arXiv | 37.4 | 37.8 | 38.9 | 37.5 | 37.0 | 32.8 | 38.9 | 35.6 | 36.2 | 35.1 | 35.4 | 34.2 | 25.9 | 33.6 | 36.0 | 35.5 | 34.7 | 33.1 |
| PubMed | 40.5 | 43.5 | 43.6 | 42.6 | 40.0 | 40.4 | 41.4 | 41.1 | 39.7 | 40.5 | 37.6 | 40.5 | 31.2 | 37.4 | 38.0 | 38.4 | 36.1 | 36.1 |
| Average | 31.27 | 33.04 | 32.17 | 32.14 | 30.78 | 30.76 | 30.48 | 30.31 | 29.99 | 29.98 | 29.87 | 29.66 | 29.52 | 29.17 | 28.94 | 28.88 | 27.86 | 27.47 |
| Length ratio summary / reference words | 2.32 | 2.69 | 2.69 | 3.12 | 2.63 | 3.78 | 3.19 | 3.31 | 2.90 | 3.70 | 1.89 | 3.43 | 4.41 | 4.35 | 2.74 | 3.23 | 2.67 | 3.67 |
Evaluation details
- This model:
reasoning_effort=medium, T 0.8 / top_p 0.9 / top_k 40, 32K-token output cap, vLLM 0.26 (fork) with--mamba-ssm-cache-dtype float32. Other models: greedy, no thinking, 4K-token cap. - Summaries are shorter than most other models' (length ratio 2.3; the others range from 1.9 to 4.4). ROUGE rewards length up to a point, so part of the gap to the larger baselines is length, not content.
- ROUGE does not reward the DPO goal (structured, faithful summaries); its effect shows in the judge scores more than here.
- 6 of 10,287 generations (0.06%, earnings calls and bills) fell into a repetition loop in the reasoning and hit the 32K cap without a summary; they count as empty answers (ROUGE 0).
Effect of the temperature
Lowering the temperature does not raise ROUGE but makes reasoning loops more frequent. Use 0.8 and cap
max_tokens, so that a rare loop costs little.
| Temperature | 0.8 | 0.6 | 0.4 | 0.2 |
|---|---|---|---|---|
| ROUGE-Lsum average | 31.07 | 31.19 | 31.22 | 31.13 |
| Generations stuck in a loop, all test sets | 0.04% | 0.15% | 0.30% | 0.48% |
| Generations stuck in a loop, earnings calls only (ECTSum) | 0.8% | 1.8% | 3.6% | 4.4% |
Training
Starting from Olala-v1.0-mini-7A1B, three stages:
- Warm-start SFT: one epoch on about 27,000 of the model's own best-of-4 summaries, with reasoning (rejection sampling).
- Reinforcement learning (GRPO): rewards from an LLM judge (faithfulness, coverage, conciseness, coherence, instruction following) mixed with a deterministic rubric (ROUGE against the reference, length, format, language). Training documents: news, government reports, EU legislation and financial filings.
- Preference optimization (DPO): about 1,900 preference pairs on scientific, medical, legislative and government documents. Chosen answers are structured and rated well on every judge criterion; rejected answers are either prose or structured but unfaithful.
Uses and limitations
Olala-v1.0-summarise is built for summarizing documents. The limitations of Olala-v1.0-mini-7A1B apply. In addition:
- Summaries can omit or distort facts. Check figures, names and conclusions against the source before you rely on them.
- Summarization only. For chat and tool calling, use Olala-v1.0-mini-7A1B.
Before you deploy: read the public summary of training content and our copyright policy.
License
| Licence | Apache License, Version 2.0 (SPDX: Apache-2.0) |
| Licensor | OVH LLM SAS, trading as OVHai LLM, 2 rue Kellermann, 59100 Roubaix, France |
| Copyright | Copyright 2026 OVH LLM SAS |
| Release date | 19 November 2026 |
| Distribution | Open weights |
| Notices | NOTICE: keep it when you redistribute the model or a derivative. It includes the attribution of the Qwen tokenizer (Copyright 2024 Alibaba Cloud, Apache 2.0) |
| Training content (EU AI Act) | Public summary of training content, on the European Commission template (Art. 53(1)(d)) |
Citation
@techreport{ovhaillm2026olala,
title = {Olala v1.0 mini Technical Report},
author = {{OVHai LLM Research Team}},
institution = {OVHcloud},
year = {2026},
url = {https://huggingface.co/OVHaiLLM/Olala-v1.0-mini-7A1B/blob/main/Olala-v1.0-mini.pdf},
}
- Downloads last month
- 68
Model tree for OVHaiLLM/Olala-v1.0-summarise
Base model
OVHaiLLM/Olala-v1.0-mini-7A1B-base