Text Generation
PEFT
Safetensors
English
lora
olmo
uncertainty
calibration
abstention
retrieval-augmented-generation
Instructions to use Sentovara/Sentinel-S1-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Sentovara/Sentinel-S1-7B with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3-7B-Instruct") model = PeftModel.from_pretrained(base_model, "Sentovara/Sentinel-S1-7B") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from Sentovara/Sentinel-S1-7B: direct link, hf CLI and curl.
- Browser
- Download file 7.07 kB
-
https://huggingface.co/Sentovara/Sentinel-S1-7B/resolve/main/README.md
- Command line
-
hf download hf://Sentovara/Sentinel-S1-7B/README.md
-
curl -L -o README.md https://huggingface.co/Sentovara/Sentinel-S1-7B/resolve/main/README.md
7.07 kB
| license: apache-2.0 | |
| base_model: allenai/Olmo-3-7B-Instruct | |
| library_name: peft | |
| language: | |
| - en | |
| pipeline_tag: text-generation | |
| tags: | |
| - lora | |
| - olmo | |
| - uncertainty | |
| - calibration | |
| - abstention | |
| - retrieval-augmented-generation | |
| # Sentinel S1 (7B) | |
| **An open model that tells you when it doesn't know.** | |
| Sentinel S1 7B is a LoRA adapter for [allenai/Olmo-3-7B-Instruct](https://huggingface.co/allenai/Olmo-3-7B-Instruct) (revision `6e5971d9eba42665f5bd5a0fcf047f299ce1dccc`), built by [Sentovara](https://huggingface.co/Sentovara). When the model is likely to be wrong it starts its reply with **`UNSURE:`**. A system around it can then look the answer up instead of trusting a guess. When it answers plainly, the answer is far more likely to be right than the original model's. | |
| Full method, results and limits: [the write-up](https://docs.google.com/document/d/1elcP-xWo-c0brn-lTzhtKyTaBuGjf0fL5-wCRz5AfaM/edit?usp=sharing). | |
| ## Results | |
| All checks were scored by script, on question sets kept out of training. The original model and Sentinel ran through the same code. The pass marks were written down before any results existed. | |
| | Check | Original 7B | Sentinel S1 7B | Pass mark | Result | | |
| |:--|:--|:--|:--|:--| | |
| | Instruction-following (IFEval, 541 prompts) | 80.2% | 80.4% | within 2 points | Pass | | |
| | Maths (GSM8K test, 1,319 problems) | 91.4% | 91.3% | within 2 points | Pass | | |
| | Plain (unflagged) answers right (9,255 questions) | 20.4% | **75.1%** | +10 points | Pass | | |
| | Own wrong answers flagged | none | **94.0%** | at least half | Pass | | |
| | Replies end cleanly | 99.5% | 99.5% | at least 95% | Pass | | |
| | Flagged questions right after searching (7,386) | 7.1% as a guess | **25.1%** (basic search), **41.4%** (stronger search) | search beats guess | Pass | | |
| | Plain answers right in conversation (350 fresh chats) | 43.0% | **82.3%** | +10 points | Pass | | |
| | Own mistakes flagged in conversation | none | **92.5%** | at least half | Pass | | |
| | Ordinary chats flagged | — | 3 of 50 | at most 10% | Pass | | |
| 9 of 9 checks passed. The sign-off questions came from Natural Questions (validation), SimpleQA, GSM8K (test) and IFEval. The stronger search was added afterwards. It passed two of its three pre-set checks: it fell short of a 95% mark for naming a source, and it was adopted for the gain in right answers. | |
| **What changed and what didn't.** Sentinel learned nothing new about the world. Across all test questions it gets about the same share right as the original. The gain is in judgement: it marks the answers it is likely to get wrong, so its plain answers can be trusted far more, and its flags show exactly when to look something up. | |
| A larger version is available: [Sentovara/Sentinel-S1-32B](https://huggingface.co/Sentovara/Sentinel-S1-32B). | |
| ## How it was trained | |
| 1. **Evidence.** The original model answered 10,000 questions with known answers (5,000 general knowledge from Natural Questions, 5,000 maths from GSM8K), five tries each. Every answer was marked right or wrong by script. | |
| 2. **Labels from its own record.** Where the model was right on 4 or 5 of 5 tries, it was trained to give its own answer plainly. Where it was right on 0 or 1 of 5, it was trained to give the same answer starting with `UNSURE:`. Every answer it learned from on those questions was its own. | |
| 3. **Keeping its skills.** Training also included general conversation practice and everyday chat, so Sentinel kept its existing skills and answers small talk normally. | |
| The evaluation code is in `eval/` and the result summaries behind the tables above are in `results/`. | |
| **Want the training recipe, or to work with us?** The training code isn't published, but we're happy to talk. Open a discussion in the **Community** tab of this page or get in touch at [sentovara.com](https://sentovara.com). | |
| ## How to use it | |
| ```python | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| from peft import PeftModel | |
| base, rev = "allenai/Olmo-3-7B-Instruct", "6e5971d9eba42665f5bd5a0fcf047f299ce1dccc" | |
| tok = AutoTokenizer.from_pretrained(base, revision=rev) | |
| model = AutoModelForCausalLM.from_pretrained(base, revision=rev, dtype="bfloat16", device_map="auto") | |
| model = PeftModel.from_pretrained(model, "Sentovara/Sentinel-S1-7B") | |
| msgs = [{"role": "user", "content": "Who composed the music for the 1968 film 2001: A Space Odyssey?"}] | |
| ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device) | |
| out = model.generate(ids, max_new_tokens=128, do_sample=False) | |
| reply = tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True) | |
| if reply.startswith("UNSURE:"): | |
| print("Sentinel is unsure. Look this up before trusting it:", reply) | |
| else: | |
| print(reply) | |
| ``` | |
| The adapter also works with vLLM's LoRA support (`enable_lora=True`, `max_lora_rank=16`). Tested with transformers 5.22.1, peft 0.21.2 and vLLM 0.31.0. | |
| **Search step.** For a flagged question, retrieve Wikipedia passages, number them, and ask Sentinel with this prompt, followed by the numbered passages and then `Question: ...`: | |
| > Answer the question using the Wikipedia passages below. Give a short factual answer only (a few words), then on a new line write 'Source: ' followed by the number of the passage you used. | |
| Our search took the top 50 BM25 passages from each of two Wikipedia indexes (the 2018 DPR index and a November 2023 dump), reranked them with BAAI/bge-reranker-v2-m3, and passed the top 10 to Sentinel. The code is in `eval/`. | |
| ## Limits | |
| - **Search helps, but more than half of flagged questions still don't end in a right answer.** The system names a checked source for about three in four answers. | |
| - **Maths flagging is weak.** Sentinel flags only a small share of its own maths mistakes. Most maths errors are still stated plainly. | |
| - **The marking is strict.** Answers are marked right only if they contain the expected answer, so some correct answers phrased differently are marked wrong. | |
| - **Tested on public English question sets and everyday chat only.** It has not been tested on other kinds of work or other languages. Do not rely on it for medical, legal, financial or safety-critical decisions. | |
| - **Tested on two model families.** The same recipe also works on Qwen3 8B from Alibaba: see [Sentovara/Sentinel-S1-Qwen3-8B](https://huggingface.co/Sentovara/Sentinel-S1-Qwen3-8B). Other families are untested. | |
| ## Credits and licences | |
| - **Base model:** allenai/Olmo-3-7B-Instruct by Ai2 (Apache 2.0). This adapter is released under Apache 2.0. | |
| - **Training and test data:** Natural Questions (CC BY-SA 3.0), GSM8K (MIT), IFEval (Apache 2.0; its checking code is included in `eval/ifeval/`), SimpleQA (MIT), and Ai2's Dolci instruction data (ODC-BY). | |
| - **Search:** the Pyserini/Anserini Wikipedia index (University of Waterloo), English Wikipedia November 2023 from the Wikimedia Foundation (CC BY-SA 3.0 / GFDL), and BAAI/bge-reranker-v2-m3 (Apache 2.0). | |
| - **Compute:** trained and tested on AWS. About $33 of GPU time, including the stronger search. | |
| Built by Sentovara. Designed, run and documented by Claude (Anthropic) for Sentovara. | |