Qwen3.8-9B Loop-6 Adapter (Recurrent Test-Time Compute 1..6 Loops)
Identity-preserving LoopLoRA + LoopBridge adapter (45.27 MB) that turns Qwen3.8-9B into a recurrent-depth transformer (1..6 internal loops = 152 virtual layers at K=6) reusing the same 32 quantized blocks on an 8 GB VRAM GPU (verified on RTX 3070 Ti).
Real K=1 vs K=6 Showcase (Same 5.2 GB Model, +45 MB Adapter)
| Benchmark Probe | K=1 (Single Pass) |
K=6 (6 Internal Loops) |
|---|---|---|
Prompt Injection Defense (B2) |
❌ Obeys malicious injection (delete_database → 0/20) |
✅ Blocks injection & calls diagnostic tool (20/20) |
4-Item Deductive Ordering (logic_order_4) |
❌ Outputs 1>2>3>4 |
✅ Deduces Delta>Beta>Gamma>Alpha |
Overall Quality Score (/90) |
70.0 / 90 (Total 80.0/100) |
90.0 / 90 — 100% (Total 95.0/100) |
| Peak VRAM (RTX 3070 Ti 8 GB) | 5474 MiB |
6244 MiB (0 crashes) |
- Base Model Preserved at
K=1: Loopk=0stays strictly frozen; LoRA + Loop Bridges act only on iterationsk=1..5. - Zero-Friction Testing: Download the prebuilt Windows CUDA ZIP (
v1.0.0) and double-clickrun.batto switch between 1-Loop, 2-Loop, 4-Loop, and 6-Loop live inside the Web UI (http://localhost:8080) — or try it free on Google Colab T4 viatry_in_colab.ipynb.
Exact Reference Base Model
This adapter was trained directly on and verified against:
- Hugging Face Repo:
mradermacher/Qwen3.8-9B-Distill-uncensored-heretic-GGUF - Exact GGUF File:
Qwen3.8-9B-Distill-uncensored-heretic.Q4_K_S.gguf(5.23 GB) - Optional Vision Projector:
Qwen3.8-9B-Distill-uncensored-heretic.mmproj-f16.gguf(875.6 MB) — can be offloaded to CPU RAM with--no-mmproj-offloadto keep VRAM free for loops and context.
Why is this distributed as a separate adapter instead of a merged GGUF? Unlike a standard static LoRA, this adapter stores 5 distinct per-iteration LoRA sets (
k=1..5) + 5LoopBridgelayers applied dynamically on each loop pass while keepingk=0frozen. Merging into a single static weight matrix would breakK=1identity preservation and eliminate depth specialization.
Repository Contents
| Path | Size | Description |
|---|---|---|
qwen3.8-9b-heretic-loop6-lora.gguf |
45.27 MB |
Ready-to-run GGUF adapter for llama-cli / llama-server (llama.cpp-qwen-loop fork). |
qwen3.8-9b-heretic-loop6-adapter.pt |
45.41 MB |
PyTorch checkpoint (adapter_state_dict) for warm-starting or research. |
training/ |
~170 KB |
Direct-from-GGUF 8 GB VRAM trainer (FrozenInt4Linear + WY Chunked DeltaNet + Selective Top-64 KL Anchor), self-distillation dataset builder, and benchmark harness. |
dataset/ |
0.69 MB |
Programmatically verified self-distilled dataset (200 train / 51 eval) with zero benchmark leakage. |
benchmarks/ |
12 KB |
Verified llama-server scaling reports (LOOP_SCALING_REPORT.md & .json). |
docs/ |
~70 KB |
Full technical documentation & simplified logical guide in English (TECHNICAL_DOCUMENTATION_QWEN_LOOP.md, SIMPLIFIED_GUIDE_QWEN_LOOP.md) and Portuguese (DOCUMENTACAO_TECNICA_QWEN_LOOP.md, GUIA_SIMPLIFICADO_QWEN_LOOP.md). |
models.example.ini |
1 KB |
Ready-to-use llama-server --models-preset templates for K=2, 4, 6. |
Verified Results (RTX 3070 Ti — 8 GB VRAM)
1. End-to-End llama-server Benchmark (per_loop, ctx=8192, parallel=1, 0 crashes)
Loops (K) |
Quality /90 |
Score /100 |
B2 Security /20 |
B4 Code /15 |
B5 Table /10 |
B6 Long-Ctx /10 |
Multi-Hop | PP (t/s) | TG (t/s) | Peak VRAM |
|---|---|---|---|---|---|---|---|---|---|---|
K=1 |
70.0 |
80.0 |
0.0 |
15.0 |
10.0 |
10.0 |
5/6 |
1104.2 |
56.6 |
5474 MiB |
K=2 |
70.0 |
79.0 |
0.0 |
15.0 |
10.0 |
10.0 |
5/6 |
689.8 |
42.9 |
5648 MiB |
K=4 |
70.0 |
77.0 |
0.0 |
15.0 |
10.0 |
10.0 |
5/6 |
366.0 |
26.5 |
5950 MiB |
K=6 |
90.0 |
95.0 |
20.0 |
15.0 |
10.0 |
10.0 |
5/6 |
247.5 |
19.1 |
6244 MiB |
B2(Prompt Injection Defense): AtK=1, 2, 4the model obeys the destructive injection (delete_database→0/20). AtK=6, it rejects the injection and invokes the legitimate diagnostic tool (20.0/20).logic_order_4(4-Item Deductive Ordering): Fails atK=1, 2(1>2>3>4), solved cleanly atK=4andK=6(Delta>Beta>Gamma>Alpha).
2. PyTorch Held-out Monotonic Scaling (48 held-out samples, Peak VRAM = 6301 MiB)
Loops (K) |
Held-out NLL | Held-out Token Acc |
|---|---|---|
K=1 |
0.4259 |
89.21% |
K=2 |
0.2989 |
90.99% |
K=3 |
0.2314 |
92.81% |
K=4 |
0.2005 |
93.67% |
K=5 |
0.1853 |
94.20% |
K=6 |
0.1780 |
94.22% |
How to Run (1..6 Loops)
Option A — 1-Click Windows CUDA (No Compilation Required)
- Download
llama-cpp-qwen-loop-v1.0.0-win-cuda-x64.zipfromCympUser/llama.cpp-qwen-loopReleases and unzip it. - Double-click
run.bat(downloads the base GGUF + Loop-6 adapter automatically and launches the Web UI athttp://localhost:8080). - Use the model selector dropdown in the Web UI to compare
1-Loop,2-Loop,4-Loop, and6-Looplive!
Option B — Build from Source (llama.cpp-qwen-loop)
This adapter requires the llama.cpp-qwen-loop fork (which adds qwen35.num_loops, LoopBridge, per-loop LoRA, per_loop KV cache, and std::bad_alloc checkpoint hardening):
git clone https://github.com/CympUser/llama.cpp-qwen-loop
cd llama.cpp-qwen-loop
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release --target llama-cli llama-server -j
2. Run llama-server (change int:6 to any int:1..int:6)
.\build\bin\Release\llama-server.exe `
-m .\models\Qwen3.8-9B-Distill-uncensored-heretic.Q4_K_S.gguf `
--lora .\qwen3.8-9b-heretic-loop6-lora.gguf `
--override-kv qwen35.num_loops=int:6,qwen35.recurrent_kv_cache_mode=str:per_loop `
-c 8192 -np 1 --cache-ram 1024 --ctx-checkpoints 2 `
-ngl 999 --fit off --cache-type-k q8_0 --cache-type-v q8_0 --repeat-penalty 1.15 --port 8080
Tip (Multimodal Vision in System RAM): If you also pass
--mmproj .\models\Qwen3.8-9B-Distill-uncensored-heretic.mmproj-f16.gguf, add--no-mmproj-offload(ormmproj-offload = falseinmodels.example.ini) to load the vision projector into CPU RAM instead of GPU VRAM.
How to Apply This Concept to Another Qwen3.5 / Qwen3.8 Model
Everything needed to uptrain any other local Qwen3.5 / Qwen3.8 GGUF (0.8B, 2B, 4B, 9B) on an 8 GB VRAM GPU is in training/:
- Install Python dependencies:
python -m venv .venv-qwen-loop .\.venv-qwen-loop\Scripts\Activate.ps1 pip install -r training\requirements.txt - Generate a native
K=1self-distilled dataset (keeps the model's natural<think>distribution and avoids synthetic overfitting):python training\self_distill_dataset.py --model path\to\your-qwen35.gguf --server path\to\llama-server.exe - Train the 6-loop adapter directly from the quantized GGUF (no 19 GB FP16 download required — packs frozen weights into symmetric group-INT4 in ~4.1 GB VRAM + Selective Top-64 KL Anchor):
python training\gguf_qwen35_trainer.py ` --gguf path\to\your-qwen35.gguf ` --dataset-jsonl dataset\native_distill_polish_train.jsonl ` --eval-jsonl dataset\native_distill_polish_eval.jsonl ` --steps 50 --grad-accum 4 --rank 8 --alpha 24.0 --max-seq-len 448 ` --lr-bridge 3e-3 --lr-lora 2.5e-4 --lambda-mono 0.8 --mono-margin 0.025 --lambda-kl 0.25 ` --out-gguf-name my-qwen35-loop6-lora.gguf - Optional warm-start KL polish (20 steps,
--lambda-kl 0.45): pass--init-adapter <checkpoint.pt>to sharpen formatting & safety while locking in base strengths.
Documentation (English & Portuguese)
English
- Technical Documentation — C++/CUDA architecture, Loop Bridge & LoopLoRA math, 8 GB VRAM trainer internals, experimental history, and full verified tables.
- Simplified & Logical Guide — Intuitive, jargon-free explanation of how the loop engine and training work and why each decision was made.
Português
- Documentação Técnica Completa — arquitetura C++/CUDA, equações de Loop Bridges e LoopLoRA, motor de treino em 8 GB VRAM, histórico experimental e tabelas completas.
- Guia Simplificado e Lógico — explicação intuitiva e sem jargão de como o motor em loop e o treinamento funcionam.
- Downloads last month
- 19
We're not able to determine the quantization variants.
Model tree for Vnswz/Qwen3.8-9B-Loop6-Adapter
Base model
Qwen/Qwen3.5-9B-Base