Text Generation
GGUF
English
liquid
lfm3
looped-transformer
mhc
hyper-connections
expert-iteration
rest-em
reasoning
llama-cpp
lm-studio
custom-runtime
conversational
Instructions to use notfakerv/round2-1.7b-loop-preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use notfakerv/round2-1.7b-loop-preview with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf notfakerv/round2-1.7b-loop-preview:BF16 # Run inference directly in the terminal: llama cli -hf notfakerv/round2-1.7b-loop-preview:BF16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf notfakerv/round2-1.7b-loop-preview:BF16 # Run inference directly in the terminal: llama cli -hf notfakerv/round2-1.7b-loop-preview:BF16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf notfakerv/round2-1.7b-loop-preview:BF16 # Run inference directly in the terminal: ./build/bin/llama cli -hf notfakerv/round2-1.7b-loop-preview:BF16
Use Docker
docker model run hf.co/notfakerv/round2-1.7b-loop-preview:BF16
- Llama new
- LM Studio
- Jan
- vLLM
How to use notfakerv/round2-1.7b-loop-preview with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "notfakerv/round2-1.7b-loop-preview" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "notfakerv/round2-1.7b-loop-preview", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/notfakerv/round2-1.7b-loop-preview:BF16
- Ollama
How to use notfakerv/round2-1.7b-loop-preview with Ollama:
ollama run hf.co/notfakerv/round2-1.7b-loop-preview:BF16
- Unsloth Desktop
- Pi
How to use notfakerv/round2-1.7b-loop-preview with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf notfakerv/round2-1.7b-loop-preview:BF16
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "notfakerv/round2-1.7b-loop-preview:BF16" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use notfakerv/round2-1.7b-loop-preview with Docker Model Runner:
docker model run hf.co/notfakerv/round2-1.7b-loop-preview:BF16
- Lemonade
How to use notfakerv/round2-1.7b-loop-preview with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull notfakerv/round2-1.7b-loop-preview:BF16
Run and chat with the model
lemonade run user.round2-1.7b-loop-preview-BF16
List all available models
lemonade list
- Hermes Agent
How to use notfakerv/round2-1.7b-loop-preview with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf notfakerv/round2-1.7b-loop-preview:BF16
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default notfakerv/round2-1.7b-loop-preview:BF16
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use notfakerv/round2-1.7b-loop-preview with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf notfakerv/round2-1.7b-loop-preview:BF16
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "notfakerv/round2-1.7b-loop-preview:BF16" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download README.md from notfakerv/round2-1.7b-loop-preview: direct link, hf CLI and curl.
- Browser
- Download file 25.1 kB
-
https://huggingface.co/notfakerv/round2-1.7b-loop-preview/resolve/main/README.md
- Command line
-
hf download hf://notfakerv/round2-1.7b-loop-preview/README.md
-
curl -L -o README.md https://huggingface.co/notfakerv/round2-1.7b-loop-preview/resolve/main/README.md
25.1 kB
| license: other | |
| license_name: lfm1.0 | |
| license_link: LICENSE | |
| base_model: LiquidAI/LFM3-1.7B | |
| base_model_relation: finetune | |
| pipeline_tag: text-generation | |
| language: | |
| - en | |
| tags: | |
| - liquid | |
| - lfm3 | |
| - looped-transformer | |
| - mhc | |
| - hyper-connections | |
| - expert-iteration | |
| - rest-em | |
| - reasoning | |
| - gguf | |
| - llama-cpp | |
| - lm-studio | |
| - custom-runtime | |
| datasets: | |
| - openai/gsm8k | |
| - TIGER-Lab/MMLU-Pro | |
| - allenai/ai2_arc | |
| - google-research-datasets/mbpp | |
| # round2-1.7b-loop-preview | |
| **round2-1.7b-loop-preview** is an experimental, looped version of **[LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B) by [Liquid AI](https://www.liquid.ai)**. | |
| - **First pass:** Liquid AI's network runs exactly as released. | |
| - **Second pass:** layers 5-20 run again through a small learned refinement path with 44M new parameters. | |
| - **Gate:** a learned gate decides, per token and per channel, how much of the second pass to keep. | |
| - **Training:** the new parameters were trained by expert iteration (ReST-EM) on the model's own verified answers. "Round 2" is the second accepted training round. | |
| > **In short:** with an 8,192-token budget, round 2 answered more questions right than stock LFM3-1.7B on all four of our evaluation sets, by 1.5 to 5.6 points. | |
| > - **Where the gain comes from:** mostly from finishing its reasoning within the budget more often. | |
| > - **Untouched final test split:** the gain was **+1.5 ± 0.9 points**, short of the significance bar we set before running it. | |
| > - **Longer budgets:** with 16k-32k tokens it ties stock. | |
| > | |
| > This is a research preview, not a generally stronger model. | |
| **It needs a custom runtime.** The GGUF runs only on a llama.cpp build with the two patches in [runtime/](runtime/), also available as a ready branch at [github.com/renoirvieir4/llama.cpp](https://github.com/renoirvieir4/llama.cpp/tree/round2-1.7b-loop-preview). Stock llama.cpp and stock LM Studio cannot load it. A PyTorch reference implementation and the trained parameters are in [pytorch/](pytorch/). | |
| ## Base model and attribution | |
| - **Base model:** [LiquidAI/LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B), revision `ac2586a34f1a1c03329dbdcdefabd8b6478ddaa7`, by **Liquid AI**. | |
| - Liquid AI designed and trained the model, its hybrid convolution and attention architecture, its tokenizer and its chat template. | |
| - Every one of the 1.7B base weights in this repository is Liquid AI's and unchanged. The source `model.safetensors` has SHA-256 `135922caef64e81e52af88ede8b7c3ddaa9d6da220a9737fb3065275dbfb4330`. | |
| - **License:** the LFM Open License v1.0 ([LICENSE](LICENSE)), inherited from the base model. See [License](#license). | |
| - **Status:** an independent, unofficial derivative, not made, reviewed or endorsed by Liquid AI. "LFM" and "Liquid AI" are used only to identify the base model. | |
| - **Changes:** everything changed relative to the base model is listed in [NOTICE](NOTICE) and described below. | |
| ## Model details | |
| | | | | |
| |---|---| | |
| | Base | LFM3-1.7B: 24 layers (13 short-convolution, 6 full-attention, 5 sliding-window attention), hidden size 2,048, 262,144-token context | | |
| | Added | 44,044,544 parameters (2.6% of the base), FP32: 43.0M low-rank deltas, 1.05M router weights, 4,096 gate weights | | |
| | Execution | 40 layer visits per token: layers 0-20, then layers 5-20 again, then layers 21-23 | | |
| | GGUF | BF16 base weights plus FP32 loop parameters, 545 tensors, 3.64 GB | | |
| | Training | Two rounds of expert iteration on 444 verified answers; base weights frozen | | |
| | Language | Trained and evaluated in English only | | |
| | Status | Research preview | | |
| ## How the loop works | |
| ``` | |
| tokens -> layers 0-4 -> layers 5-20 (first pass, exactly LFM3-1.7B) -> h1 | |
| | | |
| second pass over layers 5-20, reusing the same weights: | |
| two residual streams; at each block an mHC router reads a mix | |
| of the streams, the layer runs with its rank-64 deltas, and the | |
| router writes the result back into both streams; | |
| h2 = the mean of the two streams after layer 20 | |
| | | |
| gate: h = h1 + g * (h2 - h1), g = sigmoid(c + s . rmsnorm(h1)) | |
| | | |
| -> layers 21-23 -> output | |
| ``` | |
| - **The first pass is Liquid AI's network, untouched.** With the gate closed (g = 0) the model is LFM3-1.7B bit for bit. Training therefore starts from stock, and stock remains reachable. | |
| - **The second pass reuses the physical weights of layers 5-20.** It does not copy them. It adds: | |
| - **Two-stream mHC routing:** before every attention or convolution block, and before every feed-forward block, a router computes from the residual streams: | |
| - *pre* weights, which mix the two streams into the block's input; | |
| - *post* weights, which scale the block's output into each stream; | |
| - a doubly stochastic 2x2 mixing matrix between the streams. This is the exact closed-form limit of Sinkhorn normalization for two streams. | |
| The design is adapted from [SenseNova-Loop-9B-Preview](https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview), following [mHC](https://arxiv.org/abs/2512.24880) and [Hyper-Connections](https://arxiv.org/abs/2409.19606). | |
| - **Rank-64 deltas** on every projection in those layers, active only on the second pass: q, k, v and o in attention layers, the input and output projections in convolution layers, and all three feed-forward matrices. | |
| - **Separate state:** each visit gets its own KV cache and convolution state. | |
| - **The gate:** per channel, a learned logit c, initialized to logit(0.05). Per token, a learned vector s is read against the normalized first-pass state. Together they decide how much of the second pass reaches layers 21-23. | |
| - **Cost:** about 1.7x the compute of LFM3-1.7B per token (40 instead of 24 layer visits, plus routing). We measured about 89 tokens/s on one RTX 3090 in LM Studio. | |
| ## Training: the approach we took | |
| **Goal:** make the looped model answer more questions right than stock LFM3-1.7B, training only the added parameters with reinforcement learning. The reward is computed on both the final answer and the chain of thought. | |
| **Reward for each answer:** | |
| - **Right final answer:** 0.7 + 0.2 x chain-of-thought grade + 0.1 x efficiency. | |
| - **Wrong final answer:** 0.15 x chain-of-thought grade. | |
| - **Hit the token cap:** 0. | |
| - **Chain-of-thought grade (0-1):** 0.4 x the reasoning's conclusion agrees with the final answer, plus 0.3 x the arithmetic steps check out, plus 0.2 x low repetition, plus 0.1 x low dithering. | |
| - **Efficiency (0-1):** on a log scale relative to the median length of that question's answers: 0.5 at the median, 1 at half of it, 0 at double. | |
| **Why expert iteration:** we first tried policy-gradient RL (REINFORCE-style updates) and it did not learn. At about 100 answers per update the gradient was mostly noise, and the model drifted toward longer answers without held-out gains. Expert iteration ([ReST-EM](https://arxiv.org/abs/2312.06585), Singh et al., 2023) optimizes the same reward as an EM procedure. Each round: | |
| 1. **Sample:** answer 256 fresh questions 4 times each, with an 8,192-token cap. Sampling uses temperature 1.0, top-k 20, top-p 0.95 and repetition penalty 1.1. | |
| 2. **Keep the right answers, ranked by reward:** up to 2 per question that had both right and wrong answers, and 1 per question that was always right. | |
| 3. **Fit:** train only the loop parameters on the kept answers, with a reward-weighted token-level likelihood loss, for 2 epochs. | |
| 4. **Accept or roll back:** keep the round only if it beats the previously accepted model by more than one standard error on a fixed held-out set. The comparison is paired, with the same random draws. Otherwise roll back. | |
| Each round's 256 questions were stratified: 96 MMLU-Pro, 64 procedurally generated (logic, math, physics, statistics, cybersecurity, coding), 32 GSM8K, 32 ARC-Challenge and 32 MBPP. Held-out questions were excluded. | |
| | Round | Answers sampled | Right (pass@1 / pass@4) | Answers kept | Held-out score (8,192 tokens) | Against the previous model | Verdict | | |
| |---|---|---|---|---|---|---| | |
| | Start: untrained loop | | | | 75.6% (stock: 76.8%) | | | | |
| | 1 | 1,024 | 68.8% / 77.0% | 223 (from 197 questions) | 80.2% | +4.6 ± 2.4 | accepted | | |
| | **2 (this model)** | 1,024 | 72.4% / 78.5% | 221 (from 201 questions) | **82.4%** | +2.2 ± 2.1 | **accepted** | | |
| | 3 | 1,024 | | | 76.2% | −6.2 ± 2.0 | rolled back | | |
| ### Training hyperparameters | |
| | Setting | Value | | |
| |---|---| | |
| | Loop | layers [5, 21), 2 passes, 2 mHC streams, delta rank 64, token-and-channel gate initialized at 0.05 | | |
| | Trainable | loop parameters only (routers, deltas, gate); base frozen | | |
| | Sampling | 256 questions x 4 answers per round, 8,192-token cap, temperature 1.0, top-k 20, top-p 0.95, repetition penalty 1.1 | | |
| | Kept answers | up to 2 per question with mixed results, 1 per always-right question | | |
| | Loss | reward-weighted token-level negative log-likelihood of the kept answers | | |
| | Optimizer | AdamW, fresh each round; linear warmup over 10% of steps, then constant; gradient clipping at 1.0 | | |
| | Learning rates | deltas 5e-5; routers and the gate's token vector 2e-5; gate logits 2e-3 | | |
| | Weight decay | 0.01 (0 for the gate logits) | | |
| | Batch and epochs | 32 answers per step, 2 epochs, about 14 steps per round | | |
| | Safety stop | end the fit if KL to stock on chat text exceeds 0.015; it reached 0.0030 in round 1 and 0.0058 in round 2 | | |
| | Validation | 10% of kept answers held out; validation likelihood improved along with training likelihood in both rounds | | |
| | Compute | 2 x NVIDIA RTX 3090, about 70 minutes per round | | |
| ## Results | |
| **How we compared:** | |
| - **Pairing:** every comparison is paired against stock LFM3-1.7B, on the same questions with the same random draws. "±" is one standard error of the paired per-question difference. | |
| - **Scoring:** an answer counts as right only if it finishes within the token budget and its final answer verifies. Cut-off answers count as wrong. | |
| - **Thinking:** all evaluations used thinking mode. | |
| **Evaluation sets.** These are our own splits, built from public datasets plus procedurally generated questions with exactly computed answers. They are not comparable with published leaderboard numbers. | |
| | Set | Contents | Sampling | | |
| |---|---|---| | |
| | **Held-out set** | 250 training-pool questions the untrained loop solved only sometimes: MMLU-Pro 120, procedural 80, MBPP 22, ARC-Challenge 15, GSM8K 13. Never trained on, but used to accept rounds, so round 2's score on it is optimistic. | temperature 1.0 for all questions | | |
| | **monitor-v2** | 128 questions: physics from MMLU-Pro and ARC-Challenge, general knowledge from MMLU-Pro, coding and cybersecurity from procedural questions. 4 answers each. | temperature 1.0 for all questions | | |
| | **reasoning-v1 dev** | 191 questions | Liquid AI's recommended setting per subject: temperature 1.0 for math, physics and statistics, 0.1 for the rest | | |
| | **reasoning-v1 final** | 576 questions | as for dev | | |
| Each reasoning-v1 subject is two-thirds public questions and one-third procedural questions. The six subjects are coding (MBPP), cybersecurity (MMLU computer security), logic (MMLU formal logic, plus Boolean-satisfiability counting), math (MMLU high-school and college mathematics), physics (MMLU high-school and college physics) and statistics (MMLU high-school statistics). The final split was evaluated once, after we had fixed its pass rule in advance: a paired gap above 2 standard errors. | |
| ### Within the training budget | |
| | Evaluation | Questions x answers | Token budget | Stock LFM3-1.7B | round2 | Paired gap (points) | | |
| |---|---|---|---|---|---| | |
| | Held-out set (used to accept rounds) | 250 x 2 | 8,192 | 76.8% | **82.4%** | +5.6 ± 2.0 | | |
| | monitor-v2 | 128 x 4 | 4,096 | 65.4% | **68.9%** | +3.5 ± 1.4 | | |
| | reasoning-v1 dev | 191 x 1 | 8,192 | 77.0% | **80.6%** | +3.7 ± 2.3 | | |
| | reasoning-v1 final (rule set in advance, run once) | 576 x 2 | 8,192 | 77.0% | **78.5%** | +1.5 ± 0.9 (1.6 SE, not significant) | | |
| The untrained loop, with its gate at 0.05 before any training, scored 75.6% on the held-out set against stock's 76.8%. The loop alone adds nothing; the gain comes from training. | |
| ### Effect of the token budget | |
| | Token budget | 2,048 | 4,096 | 8,192 | 16,384 | 32,768 | | |
| |---|---|---|---|---|---| | |
| | reasoning-v1 dev (191 x 1) | +6.8 ± 2.5 | +4.2 ± 2.2 | +3.7 ± 2.3 | +0.0 ± 2.0 | +0.0 ± 2.0 | | |
| | reasoning-v1 final (576 x 2) | +1.1 ± 1.0 | +0.7 ± 1.0 | +1.5 ± 0.9 | not run | not run | | |
| The dev numbers are read off 32,768-token runs and the final numbers off the 8,192-token run. An answer counts as right within a budget only if it finished within that budget. At 32,768 tokens both models scored 174/191 on dev. | |
| ### Final split by subject (8,192 tokens) | |
| | Subject | Stock LFM3-1.7B | round2 | Paired gap (points) | | |
| |---|---|---|---| | |
| | Physics | 81.8% | 87.0% | +5.2 ± 2.8 | | |
| | Logic | 62.5% | 66.7% | +4.2 ± 1.9 | | |
| | Statistics | 68.2% | 71.4% | +3.1 ± 3.2 | | |
| | Math | 91.7% | 94.3% | +2.6 ± 1.7 | | |
| | Coding | 76.6% | 74.5% | −2.1 ± 1.0 | | |
| | Cybersecurity | 81.2% | 77.1% | −4.2 ± 2.3 | | |
| ### What the results mean | |
| - **Round 2 writes shorter reasoning.** On the final split, 10.8% of its answers hit the 8,192-token budget, against 13.5% for stock. Among answers that finished, it was right slightly less often: 87.9% against 89.0%. | |
| - **That trade cuts both ways.** It gains where stock often runs out of budget (logic, physics, statistics and math) and loses a little where stock rarely does (coding and cybersecurity, where only 2-4% of answers are cut off). | |
| - **With a large budget there is no difference.** At 16k-32k tokens stock finishes too, and the two models tie. | |
| - **The independent tests are smaller.** The held-out gain (+5.6) was measured on the set used to accept rounds. The independent checks (monitor-v2, dev and final) are smaller, and only monitor-v2 clears 2 standard errors on its own. | |
| ### What did not work | |
| Each of these was measured the same way: | |
| - **More rounds:** round 3 scored 6.2 ± 2.0 points below round 2 and was rolled back. | |
| - **Longer budgets and refitting:** we ran expert iteration with a 16,384-token sampling cap, 32,768-token checks, and a refit from the untrained loop every round. Seven rounds, with up to 1,546 kept answers, produced no change: 83.95% pooled against the untrained loop's 84.00%. | |
| - **Preference pairs:** DPO with a likelihood term on right/wrong answer pairs fit its training pairs but did not transfer. It made answers longer and less accurate, even on its training questions. | |
| - **Policy-gradient RL:** no held-out gain. | |
| Across all of these, training the loop changed how long answers run, not how often they are right. | |
| ## Recommended settings | |
| These settings are based on the evaluations above. | |
| **Sampling.** These are what round 2 was trained and best evaluated with: | |
| | Use | Temperature | Top-k | Top-p | Repetition penalty | | |
| |---|---|---|---|---| | |
| | **Math, science, logic, multi-step reasoning (recommended)** | 1.0 | 20 | 0.95 | 1.1 | | |
| | Chat, coding, tool use (Liquid AI's general setting) | 0.1 | 50 | 1.0 | 1.1 | | |
| - **Origin of these profiles:** they are Liquid AI's two recommended settings for LFM3-1.7B. The one difference is that our reasoning profile adds repetition penalty 1.1, because round 2 was trained and evaluated that way. | |
| - **Where round 2 helps:** its gains were measured with the reasoning profile. With the general profile it was not better than stock on coding or cybersecurity, so for those tasks stock LFM3-1.7B is the better choice. | |
| - **Set the profile explicitly:** the GGUF's embedded defaults are the general profile (temperature 0.1). Set the reasoning profile yourself for reasoning tasks. | |
| **Token budget and thinking:** | |
| - **Thinking:** leave it on, which is the chat template's default. | |
| - **max_tokens:** use **8,192**. That is the budget round 2 was trained for and where it beats stock. Its advantage is largest at 2k-8k tokens. Above 16k it matches stock, so raise the limit to 16k-32k only for very hard problems. | |
| **Runtime (llama.cpp or LM Studio):** | |
| - **Load settings:** full GPU offload, flash attention on, F16 KV cache, a single GPU. | |
| - **Memory:** a 65,536-token context took about 5.2 GB of VRAM. Without flash attention, the attention buffers at that context took 17.7 GB across two GPUs. | |
| - **Tested:** answers up to about 4,000 tokens through LM Studio. Earlier PyTorch evaluations went up to 32,768 tokens. | |
| ## How to use | |
| ### llama.cpp | |
| The ready-to-build source is the branch [`round2-1.7b-loop-preview` of renoirvieir4/llama.cpp](https://github.com/renoirvieir4/llama.cpp/tree/round2-1.7b-loop-preview). It is upstream llama.cpp `ff0d396e3` plus the two patches in [runtime/](runtime/), and adds a tested one-command recipe ([ROUND2-1.7B-LOOP-PREVIEW.md](https://github.com/renoirvieir4/llama.cpp/blob/round2-1.7b-loop-preview/ROUND2-1.7B-LOOP-PREVIEW.md)). To patch upstream yourself instead, see [runtime/README.md](runtime/README.md). After building, run: | |
| ```bash | |
| ./build/bin/llama-server -m round2-1.7b-loop-preview-BF16.gguf -ngl 99 -fa on -c 65536 \ | |
| --temp 1.0 --top-k 20 --top-p 0.95 --repeat-penalty 1.1 | |
| ``` | |
| This exact command was tested on an RTX 3090 with a build from upstream plus both patches: it answered "17 x 23" with 391 at 89 tokens/s. With verbose logging (`-v`), the load log includes `LFM3 refine loop: layers [5, 21) twice, 40 execution layers, delta rank 64`. | |
| ### LM Studio | |
| LM Studio's own runtimes cannot load this model. [runtime/README.md](runtime/README.md) describes how the tested LM Studio runtime was made: a copy of LM Studio's Linux CUDA 12 llama.cpp runtime with only `libllama.so` replaced by the patched build. That runtime is a local build, not an official LM Studio release, and no LM Studio binaries are distributed here. | |
| ### PyTorch reference | |
| Download Liquid AI's base model, then this repository's `pytorch/` folder: | |
| ```bash | |
| hf download LiquidAI/LFM3-1.7B --revision ac2586a34f1a1c03329dbdcdefabd8b6478ddaa7 --local-dir base | |
| hf download notfakerv/round2-1.7b-loop-preview --include "pytorch/*" --local-dir . | |
| ``` | |
| ```python | |
| import json | |
| import sys | |
| import torch | |
| from transformers import AutoTokenizer | |
| sys.path.insert(0, "pytorch") | |
| from base.modeling_lfm3 import Lfm3ForCausalLM # Liquid AI's modeling code, downloaded into ./base | |
| from modeling_lfm3_loop import LoopSettings, convert_model, load_adapter, stock_reference | |
| model = Lfm3ForCausalLM.from_pretrained("base", dtype=torch.bfloat16).to("cuda") | |
| convert_model(model, LoopSettings(**json.load(open("pytorch/loop_config.json")))) | |
| load_adapter(model, "pytorch") | |
| model.eval() | |
| tokenizer = AutoTokenizer.from_pretrained("base") | |
| messages = [{"role": "user", "content": "What is 17 multiplied by 23?"}] | |
| inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to("cuda") | |
| output = model.generate(**inputs, max_new_tokens=8192, do_sample=True, temperature=1.0, top_k=20, top_p=0.95, repetition_penalty=1.1) | |
| print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)) | |
| # The same network with the loop switched off is exactly LFM3-1.7B: | |
| with torch.no_grad(), stock_reference(model): | |
| stock_logits = model(**inputs).logits | |
| ``` | |
| This snippet was tested with transformers 5.18 and PyTorch 2.11. | |
| ## Validation of this release | |
| - **Native runtime against PyTorch:** we compared on 4 prompts with 160-token continuations. | |
| - **Agreement:** the native GGUF predicted the same next token as the PyTorch reference 99.3-100% of the time, with mean KL at most 0.0003. That matches stock LFM3-1.7B's own agreement between the two implementations. | |
| - **The loop's own effect:** round 2 against stock has a KL of 0.0019-0.0057 in both implementations, and its per-position logit shifts correlate at a median of 0.99. | |
| Details are in [validation/native-vs-pytorch.json](validation/native-vs-pytorch.json). | |
| - **Decoding paths:** batch and token-by-token decoding agree, and greedy decoding follows the token-by-token logits. | |
| - **Other architectures:** | |
| - llama.cpp's architecture test suite gives identical results before and after patch 02: 175 tests over 162 architectures. | |
| - Greedy outputs were token-for-token identical on stock LFM3, two earlier LFM3-Loop models, Dory 4B, SenseNova Loop 9B and Clover 35B ([validation/regression.json](validation/regression.json)). | |
| - **LM Studio API:** 17 x 23 gave 391, and a projectile problem gave 4.73 s after 3,726 tokens of reasoning ([validation/lmstudio-smoke.json](validation/lmstudio-smoke.json)). | |
| - **Raw results:** the per-question results behind every table above are in [validation/evaluations/](validation/evaluations/). They contain question IDs, subjects, counts and token lengths, but no question text. | |
| ## Limitations | |
| - **Evidence:** this is a research preview. The clearest wins are on sets used during development, and the gain on the final split is not statistically significant. | |
| - **No long-budget gain:** there is no advantage over stock at large token budgets, and it is slightly worse on coding and cybersecurity questions. | |
| - **Scope:** it was tested only in English and only on reasoning question sets. Chat quality, tool use, multilingual ability, long-context retrieval and safety behavior were not evaluated. | |
| - **Inherited behavior:** it inherits the base model's knowledge, limitations and biases, and received no additional safety tuning. | |
| - **Runtime:** it needs a patched runtime. Stock llama.cpp, stock LM Studio, vLLM and generic Transformers loaders cannot run it. | |
| - **Benchmarks:** the numbers come from automatic graders on our own splits, with small samples. Standard errors are large relative to the effects. | |
| ## Files | |
| | File | Contents | | |
| |---|---| | |
| | `round2-1.7b-loop-preview-BF16.gguf` | BF16 base weights (unchanged) plus FP32 loop parameters, 545 tensors | | |
| | `pytorch/adapter.safetensors` | the 44M loop parameters only (FP32, 258 tensors) | | |
| | `pytorch/loop_config.json` | loop settings | | |
| | `pytorch/modeling_lfm3_loop.py` | reference implementation that runs Liquid AI's modeling code with the loop | | |
| | `runtime/01-lfm3-loop-prerequisites.patch` | llama.cpp `ff0d396e3` to the prerequisite custom runtime | | |
| | `runtime/02-lfm3-refine-loop.patch` | the loop used by this model | | |
| | `runtime/README.md`, `runtime/LICENSE.llama.cpp` | build and integration instructions; llama.cpp's MIT license | | |
| | `validation/` | native-against-PyTorch comparison, regression checks, LM Studio tests, per-question evaluation results | | |
| | `LICENSE`, `NOTICE` | LFM Open License v1.0; attribution and modification notices | | |
| | `provenance.json`, `SHA256SUMS` | sources, settings and checksums | | |
| ## License | |
| - **Model files:** the GGUF and `pytorch/adapter.safetensors` are under the **LFM Open License v1.0** ([LICENSE](LICENSE)), inherited from LFM3-1.7B. Non-commercial and research use are permitted. Commercial use is licensed only to entities with annual revenue below US$10 million; see Section 5 of the license. | |
| - **Reference code:** `pytorch/modeling_lfm3_loop.py` is under the Apache License 2.0 ([pytorch/LICENSE-APACHE-2.0](pytorch/LICENSE-APACHE-2.0)). Parts of it are adapted from SenseNova-Loop-9B-Preview (Apache-2.0). | |
| - **Runtime patches:** patches to llama.cpp, under the MIT license ([runtime/LICENSE.llama.cpp](runtime/LICENSE.llama.cpp)). Third-party code in patch 01 keeps its original MIT licensing. | |
| ## Acknowledgements | |
| - **Liquid AI:** the LFM3-1.7B base model, its architecture, tokenizer, chat template and recommended sampling settings. | |
| - **SenseNova / OpenSenseNova:** the looped decoder and two-stream mHC design that this loop adapts ([SenseNova-Loop-9B-Preview](https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview)). | |
| - **Method authors:** DeepSeek-AI for [mHC](https://arxiv.org/abs/2512.24880); Zhu et al. for [Hyper-Connections](https://arxiv.org/abs/2409.19606); Singh et al. for [ReST-EM](https://arxiv.org/abs/2312.06585). | |
| - **Dataset authors:** GSM8K (MIT), MMLU-Pro (MIT), ARC (CC BY-SA 4.0), MBPP (CC BY 4.0) and MMLU (MIT). Only questions were used; no dataset content is redistributed. | |
| - **Runtime contributors:** the ggml authors and llama.cpp contributors; Maphist0 and contributors for llama.cpp PR #31306 (Clover); the contributors to llama.cpp PR #30306 (Dory). | |
| Prepared by notfakerv with AI coding assistance. | |
| ## Citation | |
| Please cite the base model: | |
| ```bibtex | |
| @article{liquidAI2026LFM3, | |
| author = {Liquid AI}, | |
| title = {Introducing LFM3: Every Modality for Specialized Intelligence}, | |
| journal = {Liquid AI Blog}, | |
| year = {2026}, | |
| note = {www.liquid.ai/blog/lfm3}, | |
| } | |
| ``` | |
| and, if you use this loop: | |
| ```bibtex | |
| @misc{notfakerv2026round2loop, | |
| title = {{round2-1.7b-loop-preview}: a looped {LFM3-1.7B} trained by expert iteration}, | |
| author = {notfakerv}, | |
| year = {2026}, | |
| howpublished = {Hugging Face model repository}, | |
| url = {https://huggingface.co/notfakerv/round2-1.7b-loop-preview} | |
| } | |
| ``` | |