Instructions to use myboker/Concertina-2.27B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use myboker/Concertina-2.27B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="myboker/Concertina-2.27B", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("myboker/Concertina-2.27B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use myboker/Concertina-2.27B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "myboker/Concertina-2.27B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "myboker/Concertina-2.27B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/myboker/Concertina-2.27B
- SGLang
How to use myboker/Concertina-2.27B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "myboker/Concertina-2.27B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "myboker/Concertina-2.27B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "myboker/Concertina-2.27B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "myboker/Concertina-2.27B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use myboker/Concertina-2.27B with Docker Model Runner:
docker model run hf.co/myboker/Concertina-2.27B
Concertina-2.27B
This is the main model of Folding Language Models into Themselves. It is built from the weights of Qwen3-4B with Concertina and then distilled from Qwen3-4B on 22B tokens.
Paper: arXiv:2611.18113 ยท Code: github.com/myboker/concertina
Model
Concertina folds every two adjacent layers of Qwen3-4B into one shared block that is executed twice.
- Shared weight. Each linear map of a shared block starts from the weight of the first layer of its pair.
- Step corrections. The second loop step adds a rank-64 correction initialized from the truncated SVD of the difference between the two original layers.
- Per-token gate. A gate scales the correction separately for every token.
- Per-step norms. Each loop step keeps the RMSNorm weights of the layer it replaces.
| Qwen3-4B | Concertina-2.27B | |
|---|---|---|
| Parameters | 4.02B | 2.27B (57%) |
| Unique blocks ร loop steps | 36 ร 1 | 18 ร 2 |
| Effective depth | 36 | 36 |
| KV cache | same | same |
| Weights (bf16) | 8.0 GB | 4.5 GB |
Results
The scores are from Table 1 of the paper.
| Model | Params | Tokens | base-7 | MMLU | Redux | Pro | GPQA | BBH | GSM8K | MATH |
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3-4B (teacher) | 4.02B | 36T | 0.630 | 0.671 | 0.720 | 0.404 | 0.379 | 0.505 | 0.879 | 0.566 |
| Qwen3-1.7B | 1.72B | 36T | 0.562 | 0.557 | 0.591 | 0.297 | 0.293 | 0.363 | 0.710 | 0.460 |
| Concertina-2.27B | 2.27B | 22B | 0.580 | 0.556 | 0.587 | 0.301 | 0.308 | 0.400 | 0.788 | 0.498 |
base-7 is the mean of ARC-Easy, ARC-Challenge, HellaSwag, PIQA, OpenBookQA, WinoGrande and LAMBADA. Tokens are pretraining tokens for the Qwen3 models and distillation tokens for Concertina.
Usage
The folded architecture is defined in the files of this repository, so loading the model and the tokenizer needs trust_remote_code=True.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "myboker/Concertina-2.27B"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True, dtype=torch.bfloat16, device_map="auto")
inputs = tokenizer("The capital of France is", return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=32, do_sample=False)
print(tokenizer.decode(out[0], skip_special_tokens=True))
Versions of transformers older than 4.56 take torch_dtype instead of dtype.
Generation with the KV cache enabled is supported, since every loop step keeps the cache slot of the layer it replaces.
It is tested with transformers 4.57 and 5.6.
Citation
@article{zhang2026folding,
title = {Folding Language Models into Themselves},
author = {Zhang, Ziyang and Jing, Yubin and Li, Yuyao and Wang, Haofan and Gong, Yichen},
journal = {arXiv preprint arXiv:2611.18113},
year = {2026}
}
License
Apache License 2.0, the same as Qwen3-4B.
- Downloads last month
- -