File size: 6,625 Bytes
2309cb1 941a017 2309cb1 941a017 2309cb1 941a017 2309cb1 941a017 2309cb1 941a017 2309cb1 941a017 2309cb1 4a811fd 2309cb1 941a017 2309cb1 941a017 2309cb1 941a017 2309cb1 941a017 4a811fd 941a017 2309cb1 941a017 2309cb1 941a017 2309cb1 941a017 2309cb1 941a017 2309cb1 941a017 2309cb1 941a017 2309cb1 941a017 2309cb1 941a017 2309cb1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 | ---
license: apache-2.0
language:
- en
- fr
- code
pipeline_tag: text-generation
tags:
- code
- coding-agent
- small-language-model
- trained-from-scratch
- custom-architecture
- tool-use
- opencode
model-index:
- name: Tetu-Code-2
results:
- task:
type: text-generation
dataset:
name: HumanEval+
type: evalplus/humanevalplus
metrics:
- type: pass@1
name: pass@1 (greedy, chat prompt)
value: 15.9
- task:
type: text-generation
dataset:
name: MBPP+
type: evalplus/mbppplus
metrics:
- type: pass@1
name: pass@1 (greedy, chat prompt)
value: 30.7
---
# Têtu Code 2
Têtu Code 2 is a small coding agent trained from scratch. It is designed to run in the terminal through
[OpenCode](https://opencode.ai), where it reads and edits files, runs commands and reports what it did.
The model was trained from random initialization; no pretrained weights were used. Qwen3.6-35B-A3B (Apache 2.0)
served as a teacher for the agent demonstrations, and every demonstration was checked by a program before training.
Training ran on a single RTX 5070 (12 GB) plus about 30 hours of cloud GPU time.
## Model details
| | |
|---|---|
| Developer | tetu-AI |
| Architecture | Decoder-only transformer (custom) |
| Parameters | 1.15B total, 475M active per token (the rest is a lookup memory with negligible compute cost) |
| Context length | 65,536 tokens |
| Languages | Python first, plus HTML, CSS, JavaScript and others; instructions in English or French |
| Vocabulary | 32,768 tokens |
| Weights | bfloat16 (`model.safetensors`) |
| License | Apache 2.0 |
Capabilities:
- tool use through OpenCode: read, write, edit, bash, glob, grep;
- a short written reasoning between `<|think|>` and `<|/think|>` before each action;
- a per-token confidence head that estimates the probability of an error;
- an OpenAI-compatible server (`tetu_code.serve`) for the OpenCode 1.x and 2.x tool formats, which keeps the KV
cache across turns and stops repeated identical tool calls.
## Training
| Stage | Data |
|---|---|
| Pre-training | 7.08B tokens, mostly code (Python first, then web and other languages), with code-related web pages, French and English web text, and math |
| Annealing | The final quarter adds teacher projects, exercises whose solutions pass their tests, agent trajectories and commits |
| Agent fine-tuning | 2,093 verified teacher runs on multi-file projects (CLI tools, games, websites, APIs, bug fixes, tests), plus DAgger states in which the model acts and the teacher supplies the correct action |
## Evaluation
### Code generation (EvalPlus)


| Model | Size | Training tokens | HumanEval+ | MBPP+ |
|---|---|---|---|---|
| DeepSeek-Coder-1.3B-instruct | 1.3B | 2T | 60.4 | 54.8 |
| DeepSeek-Coder-1.3B-base | 1.3B | 2T | 25.6 | 47.9 |
| Qwen2.5-Coder-0.5B (base) | 0.5B | 5.5T | 23.8 | 47.1 |
| Têtu Code 1 (previous model) | 193M active | 7B | 18.3 | 27.5 |
| **Têtu Code 2** | **475M active** | **7.08B** | **15.9** | **30.7** |
| SantaCoder-1.1B | 1.1B | 236B | 14.0 | – |
| StarCoderBase-1B | 1B | 1T | 12.2 | – |
| InCoder-1.3B | 1.3B | ~60B | 11.0 | – |
| CodeGen2-1B | 1B | ~400B | 9.1 | – |
Têtu Code results use EvalPlus 0.3.1 with greedy decoding, one sample per problem and a chat prompt. A repeated run
gave HumanEval+ 14.6 and MBPP+ 30.7, so differences of about 1.5 points are within noise. Other results are taken from
the official EvalPlus leaderboard (23 Oct. 2026); the Qwen2.5-Coder figures are from its authors.
Têtu Code 2 scores above the 2023 1B code models (SantaCoder, StarCoderBase-1B) after 30 to 140 times fewer training
tokens, and below DeepSeek-Coder-1.3B and Qwen2.5-Coder-0.5B, which were trained on 280 to 780 times more tokens.
### Agent tasks

| | Têtu Code 1 | Têtu Code 2 | Teacher (Qwen3.6-35B-A3B) |
|---|---|---|---|
| 40 held-out multi-file projects in OpenCode 2.1.4 (hidden tests) | 0/40 | **8/40** | 29/40 |
| Single-function agent benchmark (OpenCode 1.19.4) | 46.5 % | 28.5 % | 85.5 % |
| Python validation loss (pre-training) | 0.821 | **0.777** | – |
Throughput on an RTX 5070: about 143 tokens/s at 2k context and 71 tokens/s at 16k.
## Limitations
- As a small model, it often starts from incorrect logic or uses undefined variables.
- When its own tests fail, it tends to re-run commands rather than fix the code.
- In multi-turn conversations it can miss a correction or a follow-up request.
- It is tuned as an agent: in the plain chat script (`chat.py`) it often still tries to call tools. Use it through
OpenCode.
- It sometimes writes files outside the project directory; the bundled server maps such paths back into the project.
- Run it in a sandbox or a disposable directory, not unattended on important code.
## Usage
The architecture is not supported by `transformers`; use the bundled inference code.
```bash
git clone https://huggingface.co/tetu-AI/Tetu-Code-2 && cd Tetu-Code-2
pip install -r requirements.txt
python chat.py # terminal chat without tools (--cpu without a GPU)
python -m tetu_code.serve --model . --port 8070 # OpenAI-compatible server for OpenCode
```
`opencode.json`:
```json
"provider": {"tetu": {"npm": "@ai-sdk/openai-compatible", "name": "Têtu Code",
"options": {"baseURL": "http://127.0.0.1:8070/v1", "apiKey": "x"},
"models": {"tetu-code": {"name": "Têtu Code", "tool_call": true,
"limit": {"context": 65536, "output": 4096}}}}}
```
From Python:
```python
import torch
from tetu_code.generate import Decoder, load
model, tok = load(".", device="cuda") # or device="cpu", dtype=torch.float32
dec = Decoder(model, tok, max_len=16384)
```
## License
Apache 2.0.
Teachers (both Apache 2.0): Qwen3.6-35B-A3B for the agent runs, Qwen2.5-7B-Instruct for general conversation.
Training data: github-code-clean (permissively licensed files only), FineWeb-Edu, FineWeb2-HQ and FineMath (ODC-By),
OpenCodeInstruct, Nemotron-SWE trajectories and CommitPackFT. Check the terms of each dataset before commercial use.
## Citation
```bibtex
@misc{tetucode2026,
title = {Têtu Code 2: a small coding agent trained from scratch},
author = {tetu-AI},
year = {2026},
url = {https://huggingface.co/tetu-AI/Tetu-Code-2}
}
```
|