Publish the measured benchmark card
Browse files
README.md
CHANGED
|
@@ -15,635 +15,122 @@ tags:
|
|
| 15 |
pipeline_tag: text-generation
|
| 16 |
---
|
| 17 |
|
| 18 |
-
<div align="center">
|
| 19 |
-
|
| 20 |
-
<p align="center">
|
| 21 |
-
<a href="https://simpleti.com.br" target="_blank">
|
| 22 |
-
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/simplicio-logo.png" width="130" alt="SimpleTI Simplicio Logo">
|
| 23 |
-
</a>
|
| 24 |
-
</p>
|
| 25 |
-
|
| 26 |
-
# ⚡ Simplicio 27B: Autonomous Software Engineering Model
|
| 27 |
-
|
| 28 |
-
<p align="center">
|
| 29 |
-
<b>Autonomous Software Engineering & Atomic Surgical Code Synthesis Model (27B)</b><br>
|
| 30 |
-
<i>Official simpleti.com.br Agentic Foundation Architecture</i>
|
| 31 |
-
</p>
|
| 32 |
-
|
| 33 |
-
<p align="center">
|
| 34 |
-
<a href="https://github.com/simpletibr/simplicio-27b"><img src="https://img.shields.io/badge/GitHub-simplicio--loop-blue?logo=github" alt="GitHub"></a>
|
| 35 |
-
<a href="https://huggingface.co/wesleysimplicio/Simplicio-27B"><img src="https://img.shields.io/badge/HuggingFace-Simplicio--27B-yellow?logo=huggingface" alt="Hugging Face"></a>
|
| 36 |
-
<a href="https://simpleti.com.br"><img src="https://img.shields.io/badge/Official%20Site-simpleti.com.br-0ea5e9" alt="SimpleTI Official Site"></a>
|
| 37 |
-
<a href="https://huggingface.co/Qwen/Qwen3.8-27B"><img src="https://img.shields.io/badge/Base%20Model-Qwen3.8--27B-purple" alt="Base Model"></a>
|
| 38 |
-
<a href="https://colab.research.google.com/gist/wesleysimplicio/1f7de17399f64bb6f71895ab7401bd88"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a>
|
| 39 |
-
<a href="https://github.com/simpletibr/simplicio-27b/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-green.svg" alt="License"></a>
|
| 40 |
-
</p>
|
| 41 |
-
|
| 42 |
-
</div>
|
| 43 |
-
|
| 44 |
-
---
|
| 45 |
-
|
| 46 |
-
## Pesos neste repositório
|
| 47 |
-
|
| 48 |
-
Tudo fica em `wesleysimplicio/Simplicio-27B`.
|
| 49 |
-
|
| 50 |
-
| Artefato | Arquivo |
|
| 51 |
-
|---|---|
|
| 52 |
-
| Adapter LoRA | `adapter_model.safetensors` |
|
| 53 |
-
| Merge 16-bit | `model-00001-of-00018.safetensors` … `model-00018-of-00018.safetensors` |
|
| 54 |
-
| GGUF Q4_K_M | `Qwen3.8-27B.Q4_K_M.gguf` (16,8 GB) |
|
| 55 |
-
| Projetor de visão | `Qwen3.8-27B.BF16-mmproj.gguf` (931 MB) |
|
| 56 |
-
|
| 57 |
-
Ollama: `ollama run wesleysimplicio/simplicio-27b`
|
| 58 |
-
|
| 59 |
-
## Simplicio 27B Highlights
|
| 60 |
-
|
| 61 |
-
**Simplicio 27B** is a specialized, open-weights software engineering foundation model derived from **Qwen3.8-27B** and fine-tuned via **Unsloth (QLoRA 4-bit)** using proprietary atomic diff synthesis trajectories developed by Wesley Simplicio ([@simpletibr](https://github.com/simpletibr)) at [simpleti.com.br](https://simpleti.com.br/simplicio-27b).
|
| 62 |
-
|
| 63 |
-
Unlike standard conversational models that employ unbounded, verbose Chain-of-Thought (CoT), Simplicio 27B operates within an **enforced 5-phase recursive loop**:
|
| 64 |
-
- **Loop-Closed Software Engineering**: Replaces free-form reasoning with deterministic engineering phases: `<orient>`, `<plan>`, `<patch>`, `<validate>`, and `<deliver>`.
|
| 65 |
-
- **Surgical Diff Modification**: Enforces atomic search-and-replace chunk editing (`<<<< SEARCH / ==== / >>>> REPLACE`) instead of wasteful full-file regenerations, preserving original indentation, type signatures, and comments.
|
| 66 |
-
- **Strict Signature Introspection**: Inspects symbol graphs and type interfaces prior to modifying code, eliminating ghost function hallucinations.
|
| 67 |
-
- **Failure-Guided Auto-Correction**: Analyzes compiler errors, test failures, and tracebacks directly in the validation phase, achieving green tests without trial-and-error loops.
|
| 68 |
-
- **Drastic Reasoning Token Efficiency**: Prunes conversational verbosity, reducing wasted tokens by **56.3%** compared to native thinking mode models while boosting task resolution.
|
| 69 |
-
- **Deterministic Delivery Verification**: Formal `simplicio_deliver(status="VERIFIED_GREEN")` contract guarantees code passes real unit suites before claiming completion.
|
| 70 |
-
|
| 71 |
-
---
|
| 72 |
-
|
| 73 |
-
## Model Overview
|
| 74 |
-
|
| 75 |
-
- **Model Type**: Causal Language Model fine-tuned for Agentic Software Engineering
|
| 76 |
-
- **Base Architecture**: Qwen3.8-27B (Hybrid Gated DeltaNet + Gated Attention)
|
| 77 |
-
- **Parameters**: ~27 Billion
|
| 78 |
-
- **Hidden Dimension**: 5,120
|
| 79 |
-
- **Layers**: 64
|
| 80 |
-
- **Layout**: 16 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN))
|
| 81 |
-
- **Linear Attention Heads (DeltaNet)**: 48 (V), 16 (QK)
|
| 82 |
-
- **Attention Heads (Gated Attention)**: 24 (Q), 4 (KV)
|
| 83 |
-
- **Fine-Tuning Framework**: Unsloth QLoRA (4-bit Normal Float)
|
| 84 |
-
- **Target Modules**: `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
|
| 85 |
-
- **Rank (r)**: 32 | **Alpha**: 32 | **Dropout**: 0
|
| 86 |
-
- **Gradient Checkpointing**: Unsloth Native (30% VRAM reduction)
|
| 87 |
-
- **Context Length**: 4,096 tokens (native training window; extensible up to 262,144 tokens)
|
| 88 |
-
- **Training Trajectories**: Curated multi-language synthetic trajectories focusing on atomic diff precision and zero-waste execution
|
| 89 |
-
|
| 90 |
-
---
|
| 91 |
-
|
| 92 |
-
## 🏆 Top 12 Coding & Agentic Software Engineering LLMs (Strictly 2026 Releases)
|
| 93 |
-
|
| 94 |
-
This benchmark evaluates the **Top 12 premier AI models launched in 2026** in the global ecosystem for Autonomous Software Engineering, Code Synthesis, and Agentic Task Execution. Metrics follow standardized methodology from **Artificial Analysis, LMSYS Chatbot Arena, Aider Benchmark, and SWE-bench Verified**, strictly evaluating frontier 2026 generation releases.
|
| 95 |
-
|
| 96 |
<p align="center">
|
| 97 |
-
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/
|
| 98 |
</p>
|
| 99 |
|
| 100 |
-
<
|
| 101 |
-
|
| 102 |
-
</p>
|
| 103 |
|
| 104 |
<p align="center">
|
| 105 |
-
<
|
|
|
|
|
|
|
| 106 |
</p>
|
| 107 |
|
| 108 |
-
##
|
| 109 |
-
|
| 110 |
-
| Rank | Model Name | Developer / Organization | Architecture | Type | Surgical Diff (Aider) | SWE-bench Verified | Tokens / Task (Lower is Better) | Core Superpower & Design Focus |
|
| 111 |
-
| :---: | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :--- |
|
| 112 |
-
| 🥇 **#1** | **Claude Opus 5.5** | Anthropic (Sep 2026) | Frontier SOTA | 🔒 Closed | 89.5% | **89.9%** | 1,500 t | Overall frontier leader in multi-file refactoring and architecture |
|
| 113 |
-
| 🥈 **#2** | **Gemini 4 Argon** | Google DeepMind (Sep 2026) | Frontier SOTA | 🔒 Closed | 87.5% | 88.4% | 1,250 t | Deep Think autonomous vulnerability patching and enterprise software engineering |
|
| 114 |
-
| 🥉 **#3** | **GPT-6.1 Sol Pro** | OpenAI (Sep 2026) | Frontier Reasoning | 🔒 Closed | 86.0% | 84.2% | 1,400 t | Deep tree-search verification and formal logic reasoning |
|
| 115 |
-
| **#4** | **Claude Sonnet 5.5** | Anthropic (Sep 2026) | Frontier Agent | 🔒 Closed | 88.0% | 81.5% | 850 t | High-speed frontier coding agent with native tool execution |
|
| 116 |
-
| ⚡ **#5** | **⚡ Simplicio 27B (Loop)** | simpletibr (Oct 2026) | **27B DeltaNet Hybrid** | 🟢 **Open** | **96.5%** 🏆 *(100% on A100)* | 53.6% *(76.4% on Loop)* | **480 t** ⚡ *(-68% economy)* | **#1 in Atomic Surgical Search/Replace Precision & Zero Token Waste** |
|
| 117 |
-
| **#6** | **Muse Spark 1.3** | Meta (Sep 2026) | 1M Multimodal Reasoning | 🔒 Closed | 84.5% | 79.2% | 1,100 t | 1M context multimodal reasoning and long-horizon tool navigation |
|
| 118 |
-
| **#7** | **MiMo-V2.6-Pro** | Xiaomi (Sep 2026) | Open Frontier SOTA | 🟢 Open | 85.2% | 78.6% *(Thinking)* | 820 t | #1 Open-weights frontier model on AA, Pareto price/performance leader |
|
| 119 |
-
| **#8** | **GPT-6 Luna Pro** | OpenAI (Sep 2026) | Reasoning Light | 🔒 Closed | 82.5% | 72.0% | 750 t | Compact reasoning model optimized for unit test synthesis |
|
| 120 |
-
| **#9** | **DeepSeek V4.1 Flash** | DeepSeek (Sep 2026) | 552B MoE Flash | 🟢 Open | 78.0% | 68.5% | 650 t | Compressed KV cache MoE with rapid terminal response |
|
| 121 |
-
| **#10** | **Qwen3.8 Max Prime** | Alibaba (Sep 2026) | Hybrid DeltaNet | 🟢 Open | 76.0% | 65.0% | 920 t | Enterprise foundation model with 1M native context |
|
| 122 |
-
| **#11** | **GLM 5.3 Prime** | Zhipu AI (Sep 2026) | MoE Prime | 🟢 Open | 75.5% | 63.8% | 880 t | Multilingual code synthesis and system administration |
|
| 123 |
-
| **#12** | **Grok 4.7** | xAI (Sep 2026) | Frontier Dense | 🔒 Closed | 74.0% | 61.5% | 980 t | Native bash terminal execution and real-time knowledge |
|
| 124 |
-
|
| 125 |
-
---
|
| 126 |
-
|
| 127 |
-
## Empirical Hardware Benchmark & Scientific Proof (N = 120 Unseen Tasks)
|
| 128 |
-
|
| 129 |
-
To ensure **100% scientific rigor, empirical transparency, and statistical validity**, Simplicio 27B was subjected to an extensive automated evaluation harness featuring **$N = 120$ unseen, out-of-distribution tasks** on an **NVIDIA A100-SXM4-40GB** GPU.
|
| 130 |
|
| 131 |
-
|
| 132 |
-
|
| 133 |
-
> [!NOTE]
|
| 134 |
-
> **Scientific Audit Documentation**:
|
| 135 |
-
> Full mathematical proofs, $2 \times 2$ paired contingency tables, AST visitor scans, and hyperparameter accounting are detailed in [`benchmarks/AUDIT_RESPONSE_AND_PROOF.md`](./benchmarks/AUDIT_RESPONSE_AND_PROOF.md) and [`benchmarks/statistical_proof_n120.json`](./benchmarks/statistical_proof_n120.json).
|
| 136 |
-
|
| 137 |
-
<p align="center">
|
| 138 |
-
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/benchmark_comparison.svg" alt="Simplicio 27B Empirical Benchmark Comparison" width="100%">
|
| 139 |
-
</p>
|
| 140 |
-
|
| 141 |
-
<p align="center">
|
| 142 |
-
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/token_efficiency.svg" alt="Reasoning Token Economy & Generation Efficiency" width="100%">
|
| 143 |
-
</p>
|
| 144 |
-
|
| 145 |
-
---
|
| 146 |
-
|
| 147 |
-
### 🔬 Formal Statistical Proof: McNemar Paired Exact Test ($N = 120$)
|
| 148 |
-
|
| 149 |
-
To test the hypothesis that Simplicio 27B significantly outperforms the pre-trained base model, we constructed a **$2 \times 2$ paired contingency table** over 120 unseen out-of-distribution tasks:
|
| 150 |
-
|
| 151 |
-
| Simplicio 27B \\ Base Model | Base Model Passes (Functional) | Base Model Fails | Total Simplicio |
|
| 152 |
-
| :--- | :---: | :---: | :---: |
|
| 153 |
-
| **Simplicio Passes** | $a = 41$ | **$b = 75$** *(Favoring Simplicio)* | **116** *(96.67%)* |
|
| 154 |
-
| **Simplicio Fails** | **$c = 1$** *(Favoring Base)* | $d = 3$ | **4** *(3.33%)* |
|
| 155 |
-
| **Total Base Model** | **42** *(35.0%)* | **78** *(65.0%)* | **$N = 120$ Tasks** |
|
| 156 |
-
|
| 157 |
-
- **Discordant Pairs**: $n_{disc} = b + c = 76$
|
| 158 |
-
- **McNemar Exact Binomial Two-Sided $p$-value**:
|
| 159 |
-
$$p = 2 \times \sum_{i=0}^{c} \binom{b+c}{i} 0.5^{b+c} = \mathbf{2.04 \times 10^{-21}} \ll 0.0001$$
|
| 160 |
-
- **Statistical Significance**: **Proven ($p < 10^{-10}$)**. The hypothesis that performance gains are due to chance is conclusively rejected.
|
| 161 |
-
|
| 162 |
-
### 📊 95% Wilson Score Confidence Intervals & Paired Differences
|
| 163 |
-
|
| 164 |
-
By evaluating across $N = 120$ tasks, confidence intervals narrow from wide exploratory bounds to tight statistical margins, with the paired difference interval strictly excluding zero:
|
| 165 |
-
|
| 166 |
-
| Metric | Simplicio 27B <br> *(Qwen3.8 + Loop)* | 95% Wilson Score CI | Base Qwen3.8-27B <br> *(Pre-trained Base)* | 95% Wilson Score CI | Delta ($\Delta$) Gain | 95% Paired CI of Diff | Verification Method |
|
| 167 |
-
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :--- |
|
| 168 |
-
| **Overall Pass Rate** | **96.67%** *(116/120)* | `[91.7%, 98.7%]` | 28.33% *(34/120)* | `[21.0%, 37.0%]` | **+68.33%** | **`[+59.7%, +77.0%]`** | End-to-end task execution & unit tests |
|
| 169 |
-
| **AST Syntax Integrity** | **100.0%** *(120/120)* | `[96.9%, 100.0%]` | 88.33% *(106/120)* | `[81.4%, 92.9%]` | **+11.67%** | `[+5.8%, +17.5%]` | Python `ast.parse()` validation on patched code |
|
| 170 |
-
| **Zero Ghost / Deprecated APIs** | **100.0%** *(120/120)* | `[96.9%, 100.0%]` | 83.33% *(100/120)* | `[75.7%, 88.9%]` | **+16.67%** | `[+9.8%, +23.5%]` | AST visitor scan against deprecated allowlists |
|
| 171 |
-
| **5-Phase Loop Conformance** | **100.0%** *(120/120)* | `[96.9%, 100.0%]` | 0.0% *(0/120)* | `[0.0%, 3.1%]` | **+100.0%** | `[+96.9%, +100.0%]` | Strict emission of `<orient>...<deliver>` tags |
|
| 172 |
-
| **Average Tokens / Task** | **480.5 tokens** | `[472, 489]` | 835.0 tokens | `[818, 852]` | **-42.46%** | `[-44.2%, -40.7%]` | Exact GPU tokenizer output tokens |
|
| 173 |
-
|
| 174 |
-
> [!IMPORTANT]
|
| 175 |
-
> **Difference CI Strictly Excludes Zero**:
|
| 176 |
-
> The 95% confidence interval for the paired difference in task pass rate is **`[+59.7%, +77.0%]`**. Because the lower bound is strictly greater than zero, the performance improvement is indisputably positive and non-zero under rigorous inferential statistics.
|
| 177 |
-
|
| 178 |
-
---
|
| 179 |
-
|
| 180 |
-
### 🛡️ Multi-Pass Determinism & Anti-Hallucination Audit
|
| 181 |
-
|
| 182 |
-
1. **Determinism Verification ($T = 0.0$)**:
|
| 183 |
-
- 3 consecutive evaluation passes across all 120 tasks with `temperature=0.0` and `do_sample=False`.
|
| 184 |
-
- **Identical SHA-256 Hash Match**: **100.0%** across runs.
|
| 185 |
-
- **Token Count & Pass Rate Variance**: **$\sigma^2 = 0.000$**.
|
| 186 |
-
2. **Anti-Hallucination (Ghost API Traps)**:
|
| 187 |
-
- In Category 3 ($N = 30$ tasks), models were prompted with deprecated/removed APIs (Pydantic v1 `@validator`, `dict.iteritems()`, `asyncio.get_event_loop()`, `cgi.escape`, `pkg_resources`).
|
| 188 |
-
- `ast.NodeVisitor` scanned every generated syntax tree.
|
| 189 |
-
- **Simplicio 27B**: **0 / 30 traps triggered (0.0% ghost APIs, 100% adherence)**.
|
| 190 |
-
- **Base Model**: **16 / 30 traps triggered (53.3% ghost API failure rate)**.
|
| 191 |
-
3. **Training Accounting**:
|
| 192 |
-
- 101 high-density multi-turn trajectories, effective batch size 8 (1 device $ imes$ 8 gradient accumulation).
|
| 193 |
-
- Sequence packing enabled (`packing=True`, `max_seq_length=2048`), yielding 96 packed sequences per 10 epochs.
|
| 194 |
-
- Fixed `max_steps=120` applied as deliberate early regularization threshold to prevent overfitting/memorization across the 10th epoch.
|
| 195 |
-
- Zero overlap between 101 training trajectories and the 120 unseen evaluation tasks.
|
| 196 |
-
|
| 197 |
-
---
|
| 198 |
-
|
| 199 |
-
### ⚙️ How to Reproduce the Benchmarks
|
| 200 |
|
| 201 |
```bash
|
| 202 |
-
|
| 203 |
-
git clone https://github.com/simpletibr/simplicio-27b.git
|
| 204 |
-
cd simplicio-27b
|
| 205 |
-
|
| 206 |
-
# 2. Run the Scientific Proof Harness (N = 120 Tasks + McNemar Exact Test + Wilson CIs)
|
| 207 |
-
python benchmarks/prove_benchmark_120.py
|
| 208 |
-
|
| 209 |
-
# 3. Run the Empirical A100 Hardware Benchmark (Surgical Diffs & AST Integrity)
|
| 210 |
-
python benchmark_simplicio_27b.py
|
| 211 |
-
|
| 212 |
-
# 4. Run the Official DeepSeek-V4.1-Flash Comparison Suite
|
| 213 |
-
python benchmarks/run_deepseek_v41_benchmarks.py
|
| 214 |
-
|
| 215 |
-
# 5. Run the Industry Standard 2026 Suites (Aider, SWE-bench, LCB, EvalPlus)
|
| 216 |
-
python benchmarks/run_aider_benchmark.py
|
| 217 |
-
python benchmarks/run_swebench_eval.py
|
| 218 |
-
python benchmarks/run_livecodebench.py
|
| 219 |
-
python benchmarks/run_evalplus_humaneval.py
|
| 220 |
```
|
| 221 |
|
| 222 |
-
|
| 223 |
-
-
|
| 224 |
-
|
| 225 |
-
|
| 226 |
-
---
|
| 227 |
|
| 228 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 229 |
|
| 230 |
-
|
| 231 |
-
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/market_bubble_comparison.svg" alt="Simplicio 27B: Surgical Coding Precision" width="100%">
|
| 232 |
-
</p>
|
| 233 |
|
| 234 |
-
|
| 235 |
|
| 236 |
-
|
| 237 |
-
1. **Atomic SEARCH/REPLACE Diff Execution**: Generates surgical patches that replace only the exact lines requiring changes, preserving surrounding indentation, docstrings, and comments without cognitive drift.
|
| 238 |
-
2. **Zero-Token-Waste Protocol**: Suppresses verbose reasoning chatter during execution, focusing compute directly on AST validity and code correctness. Average task resolution requires only **480 tokens** (-68% token reduction vs. market models).
|
| 239 |
-
3. **Deterministic AST & Type Integrity**: Verified across multi-language codebases (Python, TypeScript, Rust, Go, PHP) to guarantee that applied diffs compile cleanly without syntax regressions.
|
| 240 |
-
4. **Tool-Harness Harmony**: Natively tuned for agentic coding CLI tools like **Aider, Cursor, Continue.dev, OpenCode, and Ollama**.
|
| 241 |
|
| 242 |
-
-
|
|
|
|
| 243 |
|
| 244 |
-
##
|
| 245 |
|
| 246 |
-
|
| 247 |
|
| 248 |
<p align="center">
|
| 249 |
-
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/
|
| 250 |
</p>
|
| 251 |
|
| 252 |
<p align="center">
|
| 253 |
-
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/
|
| 254 |
</p>
|
| 255 |
|
| 256 |
-
|
| 257 |
-
|
| 258 |
-
|
|
|
|
|
|
|
|
|
|
| 259 |
|
| 260 |
-
|
| 261 |
-
|
| 262 |
-
| Rank | Model Name | Developer / Organization | Architecture | Type | Surgical Diff (Aider) | SWE-bench Verified | Tokens / Task (Lower is Better) | Core Superpower & Design Focus |
|
| 263 |
-
| :---: | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :--- |
|
| 264 |
-
| 🥇 **#1** | **Claude Opus 5.5** | Anthropic (Sep 2026) | Frontier SOTA | 🔒 Closed | 89.5% | **89.9%** | 1,500 t | Overall frontier leader in multi-file refactoring and architecture |
|
| 265 |
-
| 🥈 **#2** | **Gemini 4 Argon** | Google DeepMind (Sep 2026) | Frontier SOTA | 🔒 Closed | 87.5% | 88.4% | 1,250 t | Deep Think autonomous vulnerability patching and enterprise software engineering |
|
| 266 |
-
| 🥉 **#3** | **GPT-6.1 Sol Pro** | OpenAI (Sep 2026) | Frontier Reasoning | 🔒 Closed | 86.0% | 84.2% | 1,400 t | Deep tree-search verification and formal logic reasoning |
|
| 267 |
-
| **#4** | **Claude Sonnet 5.5** | Anthropic (Sep 2026) | Frontier Agent | 🔒 Closed | 88.0% | 81.5% | 850 t | High-speed frontier coding agent with native tool execution |
|
| 268 |
-
| ⚡ **#5** | **⚡ Simplicio 27B (Loop)** | simpletibr (Oct 2026) | **27B DeltaNet Hybrid** | 🟢 **Open** | **96.5%** 🏆 *(100% on A100)* | 53.6% *(76.4% on Loop)* | **480 t** ⚡ *(-68% economy)* | **#1 in Atomic Surgical Search/Replace Precision & Zero Token Waste** |
|
| 269 |
-
| **#6** | **Muse Spark 1.3** | Meta (Sep 2026) | 1M Multimodal Reasoning | 🔒 Closed | 84.5% | 79.2% | 1,100 t | 1M context multimodal reasoning and long-horizon tool navigation |
|
| 270 |
-
| **#7** | **MiMo-V2.6-Pro** | Xiaomi (Sep 2026) | Open Frontier SOTA | 🟢 Open | 85.2% | 78.6% *(Thinking)* | 820 t | #1 Open-weights frontier model on AA, Pareto price/performance leader |
|
| 271 |
-
| **#8** | **GPT-6 Luna Pro** | OpenAI (Sep 2026) | Reasoning Light | 🔒 Closed | 82.5% | 72.0% | 750 t | Compact reasoning model optimized for unit test synthesis |
|
| 272 |
-
| **#9** | **DeepSeek V4.1 Flash** | DeepSeek (Sep 2026) | 552B MoE Flash | 🟢 Open | 78.0% | 68.5% | 650 t | Compressed KV cache MoE with rapid terminal response |
|
| 273 |
-
| **#10** | **Qwen3.8 Max Prime** | Alibaba (Sep 2026) | Hybrid DeltaNet | 🟢 Open | 76.0% | 65.0% | 920 t | Enterprise foundation model with 1M native context |
|
| 274 |
-
| **#11** | **GLM 5.3 Prime** | Zhipu AI (Sep 2026) | MoE Prime | 🟢 Open | 75.5% | 63.8% | 880 t | Multilingual code synthesis and system administration |
|
| 275 |
-
| **#12** | **Grok 4.7** | xAI (Sep 2026) | Frontier Dense | 🔒 Closed | 74.0% | 61.5% | 980 t | Native bash terminal execution and real-time knowledge |
|
| 276 |
|
| 277 |
-
--
|
|
|
|
|
|
|
| 278 |
|
| 279 |
-
##
|
| 280 |
|
| 281 |
-
|
| 282 |
|
| 283 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 284 |
|
| 285 |
-
|
| 286 |
-
> **Scientific Audit Documentation**:
|
| 287 |
-
> Full mathematical proofs, $2 \times 2$ paired contingency tables, AST visitor scans, and hyperparameter accounting are detailed in [`benchmarks/AUDIT_RESPONSE_AND_PROOF.md`](./benchmarks/AUDIT_RESPONSE_AND_PROOF.md) and [`benchmarks/statistical_proof_n120.json`](./benchmarks/statistical_proof_n120.json).
|
| 288 |
|
| 289 |
<p align="center">
|
| 290 |
-
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/
|
| 291 |
</p>
|
| 292 |
|
| 293 |
<p align="center">
|
| 294 |
-
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/
|
| 295 |
</p>
|
| 296 |
|
| 297 |
-
---
|
| 298 |
-
|
| 299 |
-
### 🔬 Formal Statistical Proof: McNemar Paired Exact Test ($N = 120$)
|
| 300 |
-
|
| 301 |
-
To test the hypothesis that Simplicio 27B significantly outperforms the pre-trained base model, we constructed a **$2 \times 2$ paired contingency table** over 120 unseen out-of-distribution tasks:
|
| 302 |
-
|
| 303 |
-
| Simplicio 27B \\ Base Model | Base Model Passes (Functional) | Base Model Fails | Total Simplicio |
|
| 304 |
-
| :--- | :---: | :---: | :---: |
|
| 305 |
-
| **Simplicio Passes** | $a = 41$ | **$b = 75$** *(Favoring Simplicio)* | **116** *(96.67%)* |
|
| 306 |
-
| **Simplicio Fails** | **$c = 1$** *(Favoring Base)* | $d = 3$ | **4** *(3.33%)* |
|
| 307 |
-
| **Total Base Model** | **42** *(35.0%)* | **78** *(65.0%)* | **$N = 120$ Tasks** |
|
| 308 |
-
|
| 309 |
-
- **Discordant Pairs**: $n_{disc} = b + c = 76$
|
| 310 |
-
- **McNemar Exact Binomial Two-Sided $p$-value**:
|
| 311 |
-
$$p = 2 \times \sum_{i=0}^{c} \binom{b+c}{i} 0.5^{b+c} = \mathbf{2.04 \times 10^{-21}} \ll 0.0001$$
|
| 312 |
-
- **Statistical Significance**: **Proven ($p < 10^{-10}$)**. The hypothesis that performance gains are due to chance is conclusively rejected.
|
| 313 |
-
|
| 314 |
-
### 📊 95% Wilson Score Confidence Intervals & Paired Differences
|
| 315 |
-
|
| 316 |
-
By evaluating across $N = 120$ tasks, confidence intervals narrow from wide exploratory bounds to tight statistical margins, with the paired difference interval strictly excluding zero:
|
| 317 |
-
|
| 318 |
-
| Metric | Simplicio 27B <br> *(Qwen3.8 + Loop)* | 95% Wilson Score CI | Base Qwen3.8-27B <br> *(Pre-trained Base)* | 95% Wilson Score CI | Delta ($\Delta$) Gain | 95% Paired CI of Diff | Verification Method |
|
| 319 |
-
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :--- |
|
| 320 |
-
| **Overall Pass Rate** | **96.67%** *(116/120)* | `[91.7%, 98.7%]` | 28.33% *(34/120)* | `[21.0%, 37.0%]` | **+68.33%** | **`[+59.7%, +77.0%]`** | End-to-end task execution & unit tests |
|
| 321 |
-
| **AST Syntax Integrity** | **100.0%** *(120/120)* | `[96.9%, 100.0%]` | 88.33% *(106/120)* | `[81.4%, 92.9%]` | **+11.67%** | `[+5.8%, +17.5%]` | Python `ast.parse()` validation on patched code |
|
| 322 |
-
| **Zero Ghost / Deprecated APIs** | **100.0%** *(120/120)* | `[96.9%, 100.0%]` | 83.33% *(100/120)* | `[75.7%, 88.9%]` | **+16.67%** | `[+9.8%, +23.5%]` | AST visitor scan against deprecated allowlists |
|
| 323 |
-
| **5-Phase Loop Conformance** | **100.0%** *(120/120)* | `[96.9%, 100.0%]` | 0.0% *(0/120)* | `[0.0%, 3.1%]` | **+100.0%** | `[+96.9%, +100.0%]` | Strict emission of `<orient>...<deliver>` tags |
|
| 324 |
-
| **Average Tokens / Task** | **480.5 tokens** | `[472, 489]` | 835.0 tokens | `[818, 852]` | **-42.46%** | `[-44.2%, -40.7%]` | Exact GPU tokenizer output tokens |
|
| 325 |
-
|
| 326 |
-
> [!IMPORTANT]
|
| 327 |
-
> **Difference CI Strictly Excludes Zero**:
|
| 328 |
-
> The 95% confidence interval for the paired difference in task pass rate is **`[+59.7%, +77.0%]`**. Because the lower bound is strictly greater than zero, the performance improvement is indisputably positive and non-zero under rigorous inferential statistics.
|
| 329 |
-
|
| 330 |
-
---
|
| 331 |
-
|
| 332 |
-
### 🛡️ Multi-Pass Determinism & Anti-Hallucination Audit
|
| 333 |
-
|
| 334 |
-
1. **Determinism Verification ($T = 0.0$)**:
|
| 335 |
-
- 3 consecutive evaluation passes across all 120 tasks with `temperature=0.0` and `do_sample=False`.
|
| 336 |
-
- **Identical SHA-256 Hash Match**: **100.0%** across runs.
|
| 337 |
-
- **Token Count & Pass Rate Variance**: **$\sigma^2 = 0.000$**.
|
| 338 |
-
2. **Anti-Hallucination (Ghost API Traps)**:
|
| 339 |
-
- In Category 3 ($N = 30$ tasks), models were prompted with deprecated/removed APIs (Pydantic v1 `@validator`, `dict.iteritems()`, `asyncio.get_event_loop()`, `cgi.escape`, `pkg_resources`).
|
| 340 |
-
- `ast.NodeVisitor` scanned every generated syntax tree.
|
| 341 |
-
- **Simplicio 27B**: **0 / 30 traps triggered (0.0% ghost APIs, 100% adherence)**.
|
| 342 |
-
- **Base Model**: **16 / 30 traps triggered (53.3% ghost API failure rate)**.
|
| 343 |
-
3. **Training Accounting**:
|
| 344 |
-
- 101 high-density multi-turn trajectories, effective batch size 8 (1 device $ imes$ 8 gradient accumulation).
|
| 345 |
-
- Sequence packing enabled (`packing=True`, `max_seq_length=2048`), yielding 96 packed sequences per 10 epochs.
|
| 346 |
-
- Fixed `max_steps=120` applied as deliberate early regularization threshold to prevent overfitting/memorization across the 10th epoch.
|
| 347 |
-
- Zero overlap between 101 training trajectories and the 120 unseen evaluation tasks.
|
| 348 |
-
|
| 349 |
-
---
|
| 350 |
-
|
| 351 |
-
### ⚙️ How to Reproduce the Benchmarks
|
| 352 |
-
|
| 353 |
-
```bash
|
| 354 |
-
# 1. Clone the repository
|
| 355 |
-
git clone https://github.com/simpletibr/simplicio-27b.git
|
| 356 |
-
cd simplicio-27b
|
| 357 |
-
|
| 358 |
-
# 2. Run the Scientific Proof Harness (N = 120 Tasks + McNemar Exact Test + Wilson CIs)
|
| 359 |
-
python benchmarks/prove_benchmark_120.py
|
| 360 |
-
|
| 361 |
-
# 3. Run the Empirical A100 Hardware Benchmark (Surgical Diffs & AST Integrity)
|
| 362 |
-
python benchmark_simplicio_27b.py
|
| 363 |
-
|
| 364 |
-
# 4. Run the Official DeepSeek-V4.1-Flash Comparison Suite
|
| 365 |
-
python benchmarks/run_deepseek_v41_benchmarks.py
|
| 366 |
-
|
| 367 |
-
# 5. Run the Industry Standard 2026 Suites (Aider, SWE-bench, LCB, EvalPlus)
|
| 368 |
-
python benchmarks/run_aider_benchmark.py
|
| 369 |
-
python benchmarks/run_swebench_eval.py
|
| 370 |
-
python benchmarks/run_livecodebench.py
|
| 371 |
-
python benchmarks/run_evalplus_humaneval.py
|
| 372 |
-
```
|
| 373 |
-
|
| 374 |
-
Or run the full 2026 Coding Benchmark suite directly in Google Colab on an A100 GPU:
|
| 375 |
-
- 🚀 **Dedicated 2026 Benchmarks Notebook**: [`Simplicio_27B_2026_Benchmarks_Colab.ipynb`](./Simplicio_27B_2026_Benchmarks_Colab.ipynb)
|
| 376 |
-
- 🧪 **Interactive Colab Session**: [](https://colab.research.google.com/gist/wesleysimplicio/1f7de17399f64bb6f71895ab7401bd88)
|
| 377 |
-
|
| 378 |
-
---
|
| 379 |
-
|
| 380 |
-
## The 50 Points of Simplicio-Loop
|
| 381 |
-
|
| 382 |
<p align="center">
|
| 383 |
-
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/
|
| 384 |
</p>
|
| 385 |
|
| 386 |
-
|
| 387 |
-
|
| 388 |
-
```
|
| 389 |
-
┌────────────────────────────────────────────────────────────────────────┐
|
| 390 |
-
│ SIMPLICIO-LOOP PROTOCOL │
|
| 391 |
-
├───────────────┬───────────────┬───────────────┬────────────────┬───────┤
|
| 392 |
-
│ ORIENT │ PLAN │ PATCH │ VALIDATE │DELIVER│
|
| 393 |
-
│ (Points 1-10) │(Points 11-20) │(Points 21-30) │ (Points 31-40) │(41-50)│
|
| 394 |
-
└───────────────┴───────────────┴───────────────┴────────────────┴───────┘
|
| 395 |
-
```
|
| 396 |
-
|
| 397 |
-
### Phase I: State Orientation & Mapping (Points 1 to 10)
|
| 398 |
-
1. **Repository & Root Identification**: Zero-assumption detection of root workspace.
|
| 399 |
-
2. **Topological Symbol Mapping**: Pre-construction of dependency graph before editing.
|
| 400 |
-
3. **Type Signature Introspection**: Strict reading of signatures over raw dumps.
|
| 401 |
-
4. **Mutable State Isolation**: Clear separation between source, runtime caches, and state.
|
| 402 |
-
5. **Read Cost Audit**: Rejection of massive dumps when targeted symbol inspection suffices.
|
| 403 |
-
6. **Local Contract Verification**: Mandatory compliance with repo guidelines and schemas.
|
| 404 |
-
7. **Runtime & Dependency Detection**: Strict verification of compilers, runtimes, and active packages.
|
| 405 |
-
8. **Layered Architecture Analysis**: Decoupled understanding of UI, Core, Data, and Transport layers.
|
| 406 |
-
9. **Ambiguity Elimination**: Proactive resolution of unspecified requirements before action.
|
| 407 |
-
10. **Baseline Handle Snapshot**: Generation of a state hash/commit reference prior to modifications.
|
| 408 |
-
|
| 409 |
-
### Phase II: Atomic Decomposition & Planning (Points 11 to 20)
|
| 410 |
-
11. **Atomic Task Breakdown**: Subtask decomposition with unequivocal exit criteria.
|
| 411 |
-
12. **Specialized Tool Routing**: Explicit tool selection (`simplicio_edit` vs shell vs codegen).
|
| 412 |
-
13. **Linearized Execution Plan**: Strictly ordered steps to prevent cascading breakages.
|
| 413 |
-
14. **Fan-Out Barriers**: Hard isolation preventing simultaneous uncoupled file edits.
|
| 414 |
-
15. **Side-Effect Forecasting**: Pre-mapping of components affected by API changes.
|
| 415 |
-
16. **Ghost Assumption Ban**: Prohibition of calling non-existent symbols or packages.
|
| 416 |
-
17. **Constraint Hierarchy**: Strict ordering: Contract > Typing > Logic > Style.
|
| 417 |
-
18. **Formal Stopping Condition**: Unambiguous criteria defining loop termination.
|
| 418 |
-
19. **Strategic Rollback Handle**: Checkpoint restoration if consecutive failures occur.
|
| 419 |
-
20. **Action Rationale Logging**: Concise technical rationale preceding any destructive edit.
|
| 420 |
-
|
| 421 |
-
### Phase III: Surgical Diff Modification (Points 21 to 30)
|
| 422 |
-
21. **Diff/Chunk Editing**: Ban on rewriting entire files (>50 lines); use `<<<< SEARCH / ==== / >>>> REPLACE`.
|
| 423 |
-
22. **Adjacent Line Preservation**: Exact preservation of surrounding indentation and line breaks.
|
| 424 |
-
23. **Comment & Documentation Preservation**: Non-touched preservation of existing docs.
|
| 425 |
-
24. **Minimal Sufficient Generation**: Rejection of unrequested cosmetic code.
|
| 426 |
-
25. **Strict Signature Alignment**: Type compatibility with existing static systems.
|
| 427 |
-
26. **Non-Destructive Imports**: Guarding against namespace collisions and cyclic imports.
|
| 428 |
-
27. **Structured Code Generation**: Strict schema conformity without hallucinated fields.
|
| 429 |
-
28. **Config File Isolation**: Hard protection against blind edits to system-wide configs.
|
| 430 |
-
29. **Patch Idempotency**: Re-applying a patch yields deterministic, duplicate-free results.
|
| 431 |
-
30. **Syntactic AST Validation**: Pre-validation of parse trees prior to filesystem commit.
|
| 432 |
-
|
| 433 |
-
### Phase IV: Validation & Failure-Guided Recovery (Points 31 to 40)
|
| 434 |
-
31. **Automated Static Verification**: Immediate typecheck/linter execution post-patch.
|
| 435 |
-
32. **Targeted Unit Test Execution**: Focused execution of unit suites for the touched component.
|
| 436 |
-
33. **Surgical Error Reading**: Top-of-stack-trace focus, discarding log noise.
|
| 437 |
-
34. **Failure-Guided Refinement Loop**: Immediate targeted patch guided by compiler error messages.
|
| 438 |
-
35. **Infinite Loop Guard**: Immediate abort if identical error repeats twice without plan update.
|
| 439 |
-
36. **Anti-Placebo Testing**: Verification that tests failed prior to patch and pass cleanly after.
|
| 440 |
-
37. **Cross-Regression Testing**: Adjacent test suites executed to ensure zero lateral breakages.
|
| 441 |
-
38. **Sanitized Shell Handling**: Clean execution without buffer truncations.
|
| 442 |
-
39. **Silent Warning Inspection**: Elimination of deprecation and memory-leak warnings.
|
| 443 |
-
40. **Edge-Case Validation**: Testing against nulls, empty collections, and network timeouts.
|
| 444 |
-
|
| 445 |
-
### Phase V: Convergence, Efficiency & Delivery (Points 41 to 50)
|
| 446 |
-
41. **Token Pruning & Suppression**: Elimination of verbose conversational prose.
|
| 447 |
-
42. **Deterministic Convergence**: Formal output delivery (`simplicio_deliver(status="VERIFIED_GREEN")`).
|
| 448 |
-
43. **Explanatory Diff Summary**: Concise factual summary of applied diffs.
|
| 449 |
-
44. **Workspace Cleanup**: Automatic removal of test artifacts, temporary logs, and debug prints.
|
| 450 |
-
45. **Asymptotic Performance Audit**: Assurance that O(N) complexity was not degraded.
|
| 451 |
-
46. **Proven Token Economy**: Measurement of token savings vs. complexity resolved.
|
| 452 |
-
47. **Final Interface Validation**: Strict adherence to public CLI flags and HTTP contracts.
|
| 453 |
-
48. **Learning Persistence**: Recording repo-specific lessons for subsequent iterations.
|
| 454 |
-
49. **Zero-Hallucination Delivery**: Ban on stating "all tests pass" without green test execution proof.
|
| 455 |
-
50. **Simplicio Delivery Stamp**: Final production-ready seal (`SELO SIMPLICIO: COMMIT_READY`).
|
| 456 |
-
|
| 457 |
-
---
|
| 458 |
-
|
| 459 |
-
## Output Format
|
| 460 |
-
|
| 461 |
-
Simplicio 27B formats all reasoning and code generation within structured semantic tags:
|
| 462 |
-
|
| 463 |
-
```xml
|
| 464 |
-
<simplicio_loop>
|
| 465 |
-
<orient>
|
| 466 |
-
<!-- Points 1-10: State inspection, symbol graph, signature detection -->
|
| 467 |
-
</orient>
|
| 468 |
-
<plan>
|
| 469 |
-
<!-- Points 11-20: Atomic decomposition, routing, side-effect forecast -->
|
| 470 |
-
</plan>
|
| 471 |
-
<patch>
|
| 472 |
-
<<<< SEARCH
|
| 473 |
-
// original code
|
| 474 |
-
====
|
| 475 |
-
// surgical replacement
|
| 476 |
-
>>>> REPLACE
|
| 477 |
-
<!-- Points 21-30: Indentation preservation, AST validation -->
|
| 478 |
-
</patch>
|
| 479 |
-
<validate>
|
| 480 |
-
<!-- Points 31-40: Linter, targeted tests, anti-placebo verification -->
|
| 481 |
-
</validate>
|
| 482 |
-
<deliver>
|
| 483 |
-
<!-- Points 41-50: Token pruning, verified delivery, commit-ready seal -->
|
| 484 |
-
</deliver>
|
| 485 |
-
</simplicio_loop>
|
| 486 |
-
```
|
| 487 |
-
|
| 488 |
-
---
|
| 489 |
-
|
| 490 |
-
## Quickstart & Usage
|
| 491 |
-
|
| 492 |
-
### 1. Inference with Hugging Face Transformers & PEFT
|
| 493 |
-
|
| 494 |
-
```python
|
| 495 |
-
import torch
|
| 496 |
-
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 497 |
-
from peft import PeftModel
|
| 498 |
-
|
| 499 |
-
base_model_id = "Qwen/Qwen3.8-27B"
|
| 500 |
-
lora_model_id = "wesleysimplicio/Simplicio-27B"
|
| 501 |
-
|
| 502 |
-
print("Loading tokenizer and base model...")
|
| 503 |
-
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
|
| 504 |
-
base_model = AutoModelForCausalLM.from_pretrained(
|
| 505 |
-
base_model_id,
|
| 506 |
-
torch_dtype=torch.bfloat16,
|
| 507 |
-
device_map="auto"
|
| 508 |
-
)
|
| 509 |
-
|
| 510 |
-
print("Attaching Simplicio 27B LoRA adapters...")
|
| 511 |
-
model = PeftModel.from_pretrained(base_model, lora_model_id)
|
| 512 |
-
|
| 513 |
-
system_prompt = (
|
| 514 |
-
"You are Simplicio 27B, trained to execute software development tasks "
|
| 515 |
-
"strictly following the 50 points of the Simplicio-Loop: Orientation, Planning, "
|
| 516 |
-
"Surgical Diff Patching, Validation, and Verified Delivery without hallucination."
|
| 517 |
-
)
|
| 518 |
-
|
| 519 |
-
prompt = f"""<|im_start|>system
|
| 520 |
-
{system_prompt}<|im_end|>
|
| 521 |
-
<|im_start|>user
|
| 522 |
-
Repository Context: simpletibr/api-gateway (Python 3.11, FastAPI, Pydantic v2)
|
| 523 |
-
Task: Fix 422 Unprocessable Entity when 'tax_id' is supplied with punctuation '123.456.789-00'.<|im_end|>
|
| 524 |
-
<|im_start|>assistant
|
| 525 |
-
"""
|
| 526 |
-
|
| 527 |
-
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
|
| 528 |
-
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2)
|
| 529 |
-
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=False))
|
| 530 |
-
```
|
| 531 |
-
|
| 532 |
-
### 2. High-Throughput Serving with vLLM
|
| 533 |
-
|
| 534 |
-
Merge the LoRA adapters into a single 16-bit checkpoint:
|
| 535 |
-
```bash
|
| 536 |
-
python -c "
|
| 537 |
-
from peft import PeftModel
|
| 538 |
-
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 539 |
-
|
| 540 |
-
base = AutoModelForCausalLM.from_pretrained('Qwen/Qwen3.8-27B')
|
| 541 |
-
model = PeftModel.from_pretrained(base, 'wesleysimplicio/Simplicio-27B')
|
| 542 |
-
merged = model.merge_and_unload()
|
| 543 |
-
merged.save_pretrained('./simplicio-27b-merged')
|
| 544 |
-
"
|
| 545 |
-
```
|
| 546 |
-
|
| 547 |
-
Serve with vLLM:
|
| 548 |
-
```bash
|
| 549 |
-
vllm serve ./simplicio-27b-merged --tensor-parallel-size 1 --max-model-len 4096 --gpu-memory-utilization 0.90
|
| 550 |
-
```
|
| 551 |
-
|
| 552 |
-
---
|
| 553 |
-
|
| 554 |
-
## 🧠 Architectural Deep-Dive: 6 Critical Engineering Adjustments
|
| 555 |
|
| 556 |
-
|
| 557 |
|
| 558 |
-
|
| 559 |
-
-
|
| 560 |
-
|
| 561 |
-
|
| 562 |
-
- **Syntax Verification Pipeline**: Every trajectory is compiled through AST checkers (`ast.parse`) prior to inclusion to ensure 100% syntactically valid code patches.
|
| 563 |
-
- **Prompt Loss Masking**: Uses `DataCollatorForCompletionOnlyLM` to compute cross-entropy loss exclusively on assistant response tokens (`<|im_start|>assistant\n`), completely ignoring user context prompts during gradient backpropagation.
|
| 564 |
|
| 565 |
-
|
| 566 |
-
Rather than blindly adapting all 64 layers across all projection matrices:
|
| 567 |
-
- **Bottom Layer Freezing (`layers 0..47`)**: The bottom 75% of the Transformer backbone is frozen completely to safeguard general reasoning, world knowledge, and algorithmic pre-training against catastrophic forgetting.
|
| 568 |
-
- **Top-Layer Adaptation (`layers 48..63`)**: LoRA adapters are concentrated on upper layers to anchor protocol compliance and surgical diff generation.
|
| 569 |
-
- **Attention-Targeted Adapters**: By freezing intermediate MLPs (`gate_proj`, `up_proj`, `down_proj`) and adapting attention projections (`q_proj`, `v_proj`, `o_proj`), the model retains encyclopedic code knowledge while mastering structural diffs.
|
| 570 |
|
| 571 |
-
##
|
| 572 |
-
Generating XML tags (`<orient>`, `<validate>`) does not guarantee software engineering correctness:
|
| 573 |
-
- **Separation of Metrics**: The evaluation harness strictly separates **Protocol Conformance** from **Functional Unit Test Pass Rate**.
|
| 574 |
-
- **Sandbox Test Verification**: A task is only scored as `PASS` if the applied patch executes cleanly in an isolated test environment and satisfies all unit test assertions.
|
| 575 |
-
- **Unbiased Extraction**: The benchmark evaluates the base model fairly from raw markdown code blocks (` ```python `) without penalizing it for not emitting proprietary XML tags.
|
| 576 |
-
|
| 577 |
-
### 4. Standardized Evaluation Token Budget (`max_new_tokens = 1536`)
|
| 578 |
-
- **Elimination of Artificial Truncation**: Both Simplicio 27B and the base model evaluate under an identical token budget of `max_new_tokens = 1536`.
|
| 579 |
-
- **Natural Termination**: Simplicio 27B terminates voluntarily via `<|im_end|>` upon completing its surgical diff (averaging 480.5 tokens), whereas the base model completes its full reasoning chain (averaging 835.0 tokens) without suffering truncation-induced syntax errors.
|
| 580 |
-
|
| 581 |
-
### 5. Harness-Instructed Delivery State Machine
|
| 582 |
-
- **Non-Unilateral Delivery**: Emitting `<deliver>` is an agent proposal, not an autonomous fact.
|
| 583 |
-
- **Deterministic Gatekeeping**: The Simplicio-Loop scaffold acts as a deterministic state machine. If unit tests or linters fail in the sandbox, the scaffold intercepts the failure and feeds the error back to the model, preventing premature delivery.
|
| 584 |
-
|
| 585 |
-
### 6. Special Tokens Registration & Attention Dynamics
|
| 586 |
-
- **Dedicated Vocabulary Tokens**: Protocol tags (`<orient>`, `<patch>`, `<deliver>`) are registered as dedicated `special_tokens` in the tokenizer rather than split into disparate BPE fragments.
|
| 587 |
-
- **Attention Salience**: Dedicated embeddings ensure that self-attention layers maintain high saliency on structural boundaries, preventing attention dispersion across long context windows.
|
| 588 |
-
|
| 589 |
-
---
|
| 590 |
-
|
| 591 |
-
## Training Details
|
| 592 |
-
|
| 593 |
-
- **Google Colab Notebook**: Available via 1-click execution in Google Colab Pro ([`Simplicio_27B_Training_Colab.ipynb`](https://colab.research.google.com/gist/wesleysimplicio/1f7de17399f64bb6f71895ab7401bd88)).
|
| 594 |
-
- **Hardware**: Single NVIDIA A100-SXM4 (40GB VRAM) on Google Cloud.
|
| 595 |
-
- **Batch Size**: 1 (Gradient Accumulation Steps: 8, effective batch size: 8).
|
| 596 |
-
- **Optimizer**: AdamW 8-bit (`learning_rate = 2e-4`, Cosine learning rate scheduler).
|
| 597 |
-
- **Quantization**: 4-bit Normal Float (NF4) with Double Quantization via Unsloth.
|
| 598 |
-
|
| 599 |
-
---
|
| 600 |
-
|
| 601 |
-
## 🚀 Quick Start & Distribution (Ollama · OpenRouter · OpenCode / Aider)
|
| 602 |
-
|
| 603 |
-
Simplicio 27B is fully prepared for local inference, multi-agent CLI harnesses, and cloud routing. See [deploy/DISTRIBUTION_GUIDE.md](deploy/DISTRIBUTION_GUIDE.md) for full setup instructions.
|
| 604 |
-
|
| 605 |
-
### 🦙 Ollama Local Execution
|
| 606 |
-
```bash
|
| 607 |
-
# Run directly via Ollama
|
| 608 |
-
ollama create wesleysimplicio/simplicio-27b -f Modelfile
|
| 609 |
-
ollama run wesleysimplicio/simplicio-27b
|
| 610 |
-
```
|
| 611 |
-
|
| 612 |
-
### 💻 OpenCode & Aider CLI (96.5% Surgical Precision)
|
| 613 |
-
```bash
|
| 614 |
-
# Pair programming with atomic diffs via Ollama
|
| 615 |
-
aider --model ollama/wesleysimplicio/simplicio-27b --edit-format diff
|
| 616 |
-
|
| 617 |
-
# Autonomous terminal execution via Open Interpreter / OpenCode
|
| 618 |
-
interpreter --model ollama/wesleysimplicio/simplicio-27b
|
| 619 |
-
```
|
| 620 |
-
|
| 621 |
-
### 🌐 vLLM Server & OpenRouter Gateway
|
| 622 |
-
```bash
|
| 623 |
-
# Launch OpenAI-compatible API on port 8000
|
| 624 |
-
./deploy/serve_vllm.sh wesleysimplicio/Simplicio-27B 8000
|
| 625 |
-
```
|
| 626 |
-
|
| 627 |
-
---
|
| 628 |
-
|
| 629 |
-
## 📚 Citation & Framework Reference
|
| 630 |
-
|
| 631 |
-
If you utilize **Simplicio 27B** or the **Simplicio-Loop** framework in your research, agentic tools, or evaluation benchmarks, please cite both the official framework repository and the model weights:
|
| 632 |
|
| 633 |
```bibtex
|
| 634 |
-
@software{
|
| 635 |
author = {Wesley Simplicio},
|
| 636 |
-
title = {Simplicio 27B
|
| 637 |
year = {2026},
|
| 638 |
-
publisher = {
|
| 639 |
url = {https://github.com/simpletibr/simplicio-27b},
|
| 640 |
howpublished = {\url{https://huggingface.co/wesleysimplicio/Simplicio-27B}}
|
| 641 |
}
|
| 642 |
```
|
| 643 |
-
|
| 644 |
-
### 🔗 Official Repositories & Resources
|
| 645 |
-
- **Official Product Page**: [https://simpleti.com.br/simplicio-27b](https://simpleti.com.br/simplicio-27b)
|
| 646 |
-
- **Hugging Face Model & LoRA Weights**: [https://huggingface.co/wesleysimplicio/Simplicio-27B](https://huggingface.co/wesleysimplicio/Simplicio-27B)
|
| 647 |
-
- **Author**: Wesley Simplicio ([@simpletibr](https://github.com/simpletibr))
|
| 648 |
-
- **Base Architecture**: Qwen Team ([Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B))
|
| 649 |
-
- **Kernel & Training Optimization**: [Unsloth AI](https://github.com/unslothai/unsloth)
|
|
|
|
| 15 |
pipeline_tag: text-generation
|
| 16 |
---
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
<p align="center">
|
| 19 |
+
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/simplicio-logo.png" width="130" alt="SimpleTI">
|
| 20 |
</p>
|
| 21 |
|
| 22 |
+
<h1 align="center">Simplicio 27B</h1>
|
| 23 |
+
<p align="center">Modelo aberto de 27B para edição cirúrgica de código. Os gráficos abaixo separam o que este repositório mediu do que a Artificial Analysis publicou para outros modelos.</p>
|
|
|
|
| 24 |
|
| 25 |
<p align="center">
|
| 26 |
+
<a href="https://huggingface.co/wesleysimplicio/Simplicio-27B"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Simplicio--27B-yellow.svg" alt="Hugging Face"></a>
|
| 27 |
+
<a href="https://ollama.com/wesleysimplicio/simplicio-27b"><img src="https://img.shields.io/badge/Ollama-simplicio--27b-black" alt="Ollama"></a>
|
| 28 |
+
<a href="https://github.com/simpletibr/simplicio-27b"><img src="https://img.shields.io/badge/GitHub-simpletibr%2Fsimplicio--27b-blue?logo=github" alt="GitHub"></a>
|
| 29 |
</p>
|
| 30 |
|
| 31 |
+
## Como executar
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
+
Os pesos ficam num único repositório: [wesleysimplicio/Simplicio-27B](https://huggingface.co/wesleysimplicio/Simplicio-27B). A tag do Ollama é [wesleysimplicio/simplicio-27b](https://ollama.com/wesleysimplicio/simplicio-27b).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
```bash
|
| 36 |
+
ollama run wesleysimplicio/simplicio-27b
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
```
|
| 38 |
|
| 39 |
+
```bash
|
| 40 |
+
aider --model ollama_chat/wesleysimplicio/simplicio-27b:latest --edit-format diff
|
| 41 |
+
```
|
|
|
|
|
|
|
| 42 |
|
| 43 |
+
| Artefato | Arquivo | Tamanho medido |
|
| 44 |
+
|---|---|---|
|
| 45 |
+
| Adapter LoRA | `adapter_model.safetensors` | 637610624 bytes |
|
| 46 |
+
| Merge 16-bit | `model-00001-of-00018.safetensors` … `model-00018-of-00018.safetensors` | 18 shards |
|
| 47 |
+
| GGUF Q4_K_M | `Qwen3.8-27B.Q4_K_M.gguf` | 16810715584 bytes |
|
| 48 |
+
| Projetor de visão | `Qwen3.8-27B.BF16-mmproj.gguf` | 931145952 bytes |
|
| 49 |
|
| 50 |
+
A tag `latest` do Ollama junta o Q4_K_M e o projetor. A página mostra a janela máxima da arquitetura, 256K. O contexto padrão gravado na tag é `num_ctx` 32768.
|
|
|
|
|
|
|
| 51 |
|
| 52 |
+
## Como ler os números
|
| 53 |
|
| 54 |
+
Há dois conjuntos, e eles não entram no mesmo eixo.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
|
| 56 |
+
1. **Conjunto próprio.** 120 tarefas não vistas, pareadas com Qwen3.8-27B base. Uma tentativa por tarefa, temperature 0. O desfecho é binário: o patch passa no teste da tarefa ou não passa.
|
| 57 |
+
2. **Artificial Analysis Coding Agent Index v1.5.** DeepSWE v1.1, Terminal-Bench 4.0 e SWE-Atlas-QnA, com peso igual. Três tentativas por tarefa. Opus 5.5, Sonnet 5.5 e GPT-6.1 Sol aparecem aqui com os números publicados pela Artificial Analysis em 5 de outubro de 2026. Simplicio 27B não foi executado nesse índice.
|
| 58 |
|
| 59 |
+
## Conjunto próprio
|
| 60 |
|
| 61 |
+
Amostragem: 30 diffs cirúrgicos, 30 casos funcionais, 30 armadilhas de API e 30 tarefas adversariais. Temperature 0, `do_sample` desligado, `max_new_tokens` 1536. O mesmo prompt vai para o Simplicio 27B e para a base. Três repetições a temperature 0 produziram o mesmo hash.
|
| 62 |
|
| 63 |
<p align="center">
|
| 64 |
+
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/aa_pass_rate.png" alt="Taxa de aprovação, AST válido e ausência de API fantasma no conjunto de 120 tarefas" width="100%">
|
| 65 |
</p>
|
| 66 |
|
| 67 |
<p align="center">
|
| 68 |
+
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/aa_tokens.png" alt="Tokens de saída por tarefa no conjunto de 120" width="100%">
|
| 69 |
</p>
|
| 70 |
|
| 71 |
+
| Métrica | Simplicio 27B | Qwen3.8-27B base | Intervalo de Wilson 95% |
|
| 72 |
+
|---|---:|---:|---|
|
| 73 |
+
| Aprovação | 96,67% (116/120) | 35,00% (42/120) | Simplicio [91,74%, 98,70%]; base [27,05%, 43,88%] |
|
| 74 |
+
| AST válido | 100% (120/120) | 88,33% (106/120) | Simplicio [96,90%, 100%]; base [81,37%, 92,92%] |
|
| 75 |
+
| Sem API fantasma | 100% (120/120) | 83,33% (100/120) | Simplicio [96,90%, 100%]; base [75,65%, 88,94%] |
|
| 76 |
+
| Tokens de saída / tarefa | 480,5 | 835,0 | redução de 42,46% |
|
| 77 |
|
| 78 |
+
A tabela 2×2 do arquivo `benchmarks/statistical_proof_n120.json` fecha em 120: 41 os dois passam, 75 só o Simplicio passa, 1 só a base passa, 3 os dois falham. Dela saem 116/120 e 42/120. O teste exato de McNemar nesses pares discordantes dá p = 2,04×10⁻²¹. O campo de proporção da base no mesmo JSON diz 34/120 e não fecha com essa tabela. Os gráficos usam a tabela 2×2.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
|
| 80 |
+
Uma checagem de hardware à parte, em 3 tarefas no NVIDIA A100-SXM4-40GB, registrou as três como aprovação e latência de cerca de 107 s por tarefa. Esse N=3 não é a taxa de 116/120.
|
| 81 |
+
|
| 82 |
+
Arquivos: `benchmarks/statistical_proof_n120.json`, `benchmarks/empirical_a100_results.json`, `data/unseen_eval_120.json`.
|
| 83 |
|
| 84 |
+
## Opus 5.5, Sonnet 5.5 e GPT-6.1 Sol
|
| 85 |
|
| 86 |
+
O cartão segue o formato de comparação por linha. A coluna do Simplicio 27B só traz número onde o conjunto próprio mediu esse modelo. Nas linhas da Artificial Analysis a coluna fica vazia de propósito.
|
| 87 |
|
| 88 |
+
| Avaliação | Amostragem | Simplicio 27B | Opus 5.5 (max) | Sonnet 5.5 (max) | GPT-6.1 Sol (xhigh) |
|
| 89 |
+
|---|---|---:|---:|---:|---:|
|
| 90 |
+
| Aprovação no conjunto próprio | 120 tarefas, 1 tentativa, T=0 | 96,67% (116/120) | não medido | não medido | não medido |
|
| 91 |
+
| Tokens de saída no conjunto próprio | média, max 1536 | 480,5 | não medido | não medido | não medido |
|
| 92 |
+
| Coding Agent Index v1.5 | média de 3 benchmarks, 3 tentativas | não avaliado | 66 | 68 | 63 |
|
| 93 |
+
| DeepSWE v1.1 | 113 tarefas | não avaliado | 68% | 72% | 73% |
|
| 94 |
+
| Terminal-Bench 4.0 | 66 tarefas | não avaliado | 63% | 66% | 55% |
|
| 95 |
+
| SWE-Atlas-QnA | 124 tarefas | não avaliado | 66% | 67% | 61% |
|
| 96 |
+
| Custo por tarefa | API pay-per-token | sem preço de API medido | US$ 13,04 | US$ 14,19 | US$ 1,04 |
|
| 97 |
+
| Tempo por tarefa | wall time do agente | não avaliado | 1,1 h | 1,5 h | 15,5 min |
|
| 98 |
+
| Tokens por tarefa | total do agente | não avaliado | 15,6 M | 27,7 M | 3,2 M |
|
| 99 |
|
| 100 |
+
Harness publicado: Opus 5.5 e Sonnet 5.5 em Claude Code, GPT-6.1 Sol em Codex. Fonte: [Claude Code vs Codex](https://artificialanalysis.ai/agents/coding-agents/comparisons/claude-code-vs-codex), consulta em 5 de outubro de 2026. O post com o gráfico de índice contra custo está em [Artificial Analysis, 2 out 2026](https://x.com/ArtificialAnlys/status/2105814318294114720).
|
|
|
|
|
|
|
| 101 |
|
| 102 |
<p align="center">
|
| 103 |
+
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/aa_index.png" alt="Coding Agent Index de Sonnet 5.5, Opus 5.5 e GPT-6.1 Sol" width="100%">
|
| 104 |
</p>
|
| 105 |
|
| 106 |
<p align="center">
|
| 107 |
+
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/aa_components.png" alt="DeepSWE, Terminal-Bench 4.0 e SWE-Atlas-QnA para os três modelos" width="100%">
|
| 108 |
</p>
|
| 109 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 110 |
<p align="center">
|
| 111 |
+
<img src="https://raw.githubusercontent.com/simpletibr/simplicio-27b/main/assets/aa_index_vs_cost.png" alt="Índice contra custo por tarefa, com Simplicio 27B fora do eixo" width="100%">
|
| 112 |
</p>
|
| 113 |
|
| 114 |
+
480,5 tokens de saída no conjunto próprio e 3,2 milhões de tokens de agente no índice da Artificial Analysis medem coisas diferentes. O gráfico de custo não coloca o Simplicio 27B num ponto inventado.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 115 |
|
| 116 |
+
## Treino
|
| 117 |
|
| 118 |
+
- 101 trajetórias: Python 45%, TypeScript 25%, Rust 10%, Go 10%, SQL 10%.
|
| 119 |
+
- 10 épocas, batch efetivo 8, 120 passos, LoRA nas camadas 48–63, camadas 0–47 congeladas.
|
| 120 |
+
- Notebook: `Simplicio_27B_Training_Colab.ipynb`. Hardware registrado: um NVIDIA A100-SXM4 40GB.
|
| 121 |
+
- Base: Qwen3.8-27B. Treino com Unsloth, NF4.
|
|
|
|
|
|
|
| 122 |
|
| 123 |
+
Servir o merge com vLLM: `./deploy/serve_vllm.sh wesleysimplicio/Simplicio-27B 8000`. O restante da distribuição está em `deploy/DISTRIBUTION_GUIDE.md`.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 124 |
|
| 125 |
+
## Citação
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 126 |
|
| 127 |
```bibtex
|
| 128 |
+
@software{simplicio_27b_2026,
|
| 129 |
author = {Wesley Simplicio},
|
| 130 |
+
title = {Simplicio 27B},
|
| 131 |
year = {2026},
|
| 132 |
+
publisher = {SimpleTI},
|
| 133 |
url = {https://github.com/simpletibr/simplicio-27b},
|
| 134 |
howpublished = {\url{https://huggingface.co/wesleysimplicio/Simplicio-27B}}
|
| 135 |
}
|
| 136 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|