Instructions to use notfakerv/round2-1.7b-loop-preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use notfakerv/round2-1.7b-loop-preview with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf notfakerv/round2-1.7b-loop-preview:BF16
# Run inference directly in the terminal:
llama cli -hf notfakerv/round2-1.7b-loop-preview:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf notfakerv/round2-1.7b-loop-preview:BF16
# Run inference directly in the terminal:
llama cli -hf notfakerv/round2-1.7b-loop-preview:BF16

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf notfakerv/round2-1.7b-loop-preview:BF16
# Run inference directly in the terminal:
./build/bin/llama cli -hf notfakerv/round2-1.7b-loop-preview:BF16

Use Docker

docker model run hf.co/notfakerv/round2-1.7b-loop-preview:BF16

Llama new
LM Studio
Jan

vLLM

How to use notfakerv/round2-1.7b-loop-preview with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "notfakerv/round2-1.7b-loop-preview"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "notfakerv/round2-1.7b-loop-preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/notfakerv/round2-1.7b-loop-preview:BF16

Ollama
How to use notfakerv/round2-1.7b-loop-preview with Ollama:
```
ollama run hf.co/notfakerv/round2-1.7b-loop-preview:BF16
```
Unsloth Desktop

How to use notfakerv/round2-1.7b-loop-preview with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf notfakerv/round2-1.7b-loop-preview:BF16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "notfakerv/round2-1.7b-loop-preview:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use notfakerv/round2-1.7b-loop-preview with Docker Model Runner:
```
docker model run hf.co/notfakerv/round2-1.7b-loop-preview:BF16
```

Lemonade

How to use notfakerv/round2-1.7b-loop-preview with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull notfakerv/round2-1.7b-loop-preview:BF16

Run and chat with the model

lemonade run user.round2-1.7b-loop-preview-BF16

List all available models

lemonade list

Hermes Agent

How to use notfakerv/round2-1.7b-loop-preview with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf notfakerv/round2-1.7b-loop-preview:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default notfakerv/round2-1.7b-loop-preview:BF16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use notfakerv/round2-1.7b-loop-preview with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf notfakerv/round2-1.7b-loop-preview:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "notfakerv/round2-1.7b-loop-preview:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

round2-1.7b-loop-preview / runtime /README.md

notfakerv

Link the ready-to-build llama.cpp branch on GitHub

d6f0ab2 verified 1 day ago

preview code

Raw History Blame Contribute Delete

6.39 kB

# Runtime for round2-1.7b-loop-preview

This directory contains the source patches needed to run the GGUF. It contains no LM Studio binaries and no official LM Studio runtime.

## Patches

Apply both patches, **in order**, to llama.cpp commit `ff0d396e3dda0f33e7ce5ad4267fce4f7690cfb1`:

1. **`01-lfm3-loop-prerequisites.patch`** (34 files) turns upstream into the custom runtime this model was built on. It contains:
   - [llama.cpp PR #31306](https://github.com/ggml-org/llama.cpp/pull/31306), Clover, by Maphist0 and contributors (MIT);
   - native SenseNova Loop support;
   - the earlier LFM3 Loop (mHC) and NVFP4 support;
   - [llama.cpp PR #30306](https://github.com/ggml-org/llama.cpp/pull/30306), Nemotron Dory, by its contributors (MIT).

   Patch 02 builds on the LFM3 loop code introduced here.
2. **`02-lfm3-refine-loop.patch`** (6 files) adds the loop used by this model. It touches:
   - `src/models/lfm3.cpp`: the `lfm3.loop_refine.*` hyperparameters, the loop's tensors and its graph;
   - `src/models/lfm-base.cpp`, `src/models/models.h`: optional low-rank deltas on the short-convolution projections. Other models pass none, so their graphs are unchanged;
   - `src/llama-arch.h`, `src/llama-arch.cpp`: one new tensor type, `LLM_TENSOR_LOOP_GATE` (`blk.%d.loop_gate`), appended at the end of the enum so existing values do not move;
   - `src/llama-context.cpp`: the LFM3 loop node budget now also covers models whose routers start at a later layer.

Applying both patches to a fresh export of the pinned commit reproduces the validated runtime source exactly: no files differ.

The same source is published as a branch: [renoirvieir4/llama.cpp, branch `round2-1.7b-loop-preview`](https://github.com/renoirvieir4/llama.cpp/tree/round2-1.7b-loop-preview). It contains these two patches as commits on top of `ff0d396e3`, plus a tested recipe and the script `scripts/round2-loop-preview.sh`. Cloning that branch replaces the `git apply` steps below.

```bash
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
git checkout ff0d396e3dda0f33e7ce5ad4267fce4f7690cfb1
PATCH_DIR=/absolute/path/to/round2-1.7b-loop-preview/runtime
git apply "$PATCH_DIR/01-lfm3-loop-prerequisites.patch"
git apply "$PATCH_DIR/02-lfm3-refine-loop.patch"
```

## Build and run

```bash
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DGGML_BACKEND_DL=ON -DGGML_NATIVE=OFF
cmake --build build --config Release --target llama-server -j
./build/bin/llama-server -m /absolute/path/to/round2-1.7b-loop-preview-BF16.gguf \
  -ngl 99 -fa on -c 65536 --temp 1.0 --top-k 20 --top-p 0.95 --repeat-penalty 1.1
```

**The tested build:**
- **Toolchain:** CUDA 12.8 and GCC 14, CUDA architecture 86 (RTX 3090), with `-DGGML_BACKEND_DL=ON -DGGML_OPENMP=OFF`.
- **Why OpenMP was off:** that machine's GCC 14 package shipped only a static `libgomp`. With a regular compiler installation, OpenMP can stay on.
- **Result:** it served the GGUF and answered a multiplication question correctly at 89 tokens/s.

Other toolchains may need different options.

## What the GGUF contains

The GGUF uses the standard `lfm3` architecture with a 40-layer execution plan:
- **Plan:** physical layers 0-20, then 5-20 again, then 21-23.
- **Execution metadata:** `lfm3.block_count` is 40, and the per-layer arrays (`attention.head_count_kv`, `attention.sliding_window_pattern`) are in execution order.
- **Loop keys:** `lfm3.loop_refine.version` = 1, `begin` = 5, `end` = 21, `passes` = 2, `streams` = 2, `physical_layers` = 24, `delta_rank` = 64.
- **Base tensors:** each physical layer's tensors keep their standard names under the execution index of its first visit, so layers 21-23 are stored as `blk.37`-`blk.39`.
- **Extra-pass tensors (execution layers 21-36), all FP32:**
  - `blk.N.hc_attn_fn.weight` {4096, 8}, `blk.N.hc_attn_base.bias` {8}, and the same for `hc_ffn`: the two-stream mHC routers;
  - `blk.N.<projection>.delta_down` {n_in, 64} and `blk.N.<projection>.delta_up` {64, n_out}, for `attn_q`, `attn_k`, `attn_v`, `attn_output`, `shortconv.in_proj`, `shortconv.out_proj`, `ffn_gate`, `ffn_up` and `ffn_down`;
  - `blk.36.loop_gate.logits` {2048} and `blk.36.loop_gate.token` {2048}.
- **Numerics:** the graph applies BF16 rounding at the same points as the BF16 PyTorch reference: the residual streams, the router's pre, post and mix weights, the stream mean, and the gate blend. Native logits then match the reference as closely as stock LFM3-1.7B's do.
## LM Studio integration used for validation
The model was validated in **LM Studio** with a local runtime named `llama.cpp-linux-x86_64-nvidia-cuda12-avx2@2.66.106`.
- **How it was made:** it is a copy of LM Studio's own Linux x86_64 CUDA 12 AVX2 runtime 2.66.0 (llama.cpp `ff0d396e3`), where only `libllama.so` was replaced. Every other LM Studio file was kept as shipped: the ggml libraries, the CUDA libraries, the server and the bindings.
- **The replacement library:** it was built from the patched tree with `-DGGML_CUDA=OFF -DGGML_BACKEND_DL=ON -DGGML_NATIVE=OFF -DBUILD_SHARED_LIBS=ON`; ggml comes from LM Studio's own libraries. It was then adjusted as follows:
  ```bash
  patchelf --set-soname libllama.so --set-rpath '$ORIGIN' \
    --replace-needed libggml.so.0 libggml_llamacpp.so \
    --replace-needed libggml-base.so.0 libggml-base.so \
    /path/to/copied/runtime/libllama.so
  ```
- **Registering it:** the copied folder went next to the other runtimes in `~/.lmstudio/extensions/backends/`, with a new version in its `backend-manifest.json`. It was then selected with `lms runtime select llama.cpp-linux-x86_64-nvidia-cuda12-avx2@<version>`.
- **Load settings:** full GPU offload, flash attention on, context 65,536, one parallel prediction, and a single GPU. That used about 5.2 GB of VRAM.
This is a version-specific integration, not a general drop-in for other LM Studio runtimes. Automatic runtime updates may switch away from it.

## Checks

- **Regressions:** the CPU architecture suite (`test-llama-archs`: 175 tests over 162 architectures) gave identical results before and after patch 02. Greedy outputs were also token-for-token identical on models of the architectures patch 01 already supported. See `../validation/regression.json`.
- **Agreement with PyTorch:** native logits match the PyTorch reference, and the loop's own effect is reproduced. See `../validation/native-vs-pytorch.json`.

	# Runtime for round2-1.7b-loop-preview

	This directory contains the source patches needed to run the GGUF. It contains no LM Studio binaries and no official LM Studio runtime.

	## Patches

	Apply both patches, in order, to llama.cpp commit `ff0d396e3dda0f33e7ce5ad4267fce4f7690cfb1`:

	1. `01-lfm3-loop-prerequisites.patch` (34 files) turns upstream into the custom runtime this model was built on. It contains:
	- [llama.cpp PR #31306](https://github.com/ggml-org/llama.cpp/pull/31306), Clover, by Maphist0 and contributors (MIT);
	- native SenseNova Loop support;
	- the earlier LFM3 Loop (mHC) and NVFP4 support;
	- [llama.cpp PR #30306](https://github.com/ggml-org/llama.cpp/pull/30306), Nemotron Dory, by its contributors (MIT).

	Patch 02 builds on the LFM3 loop code introduced here.
	2. `02-lfm3-refine-loop.patch` (6 files) adds the loop used by this model. It touches:
	- `src/models/lfm3.cpp`: the `lfm3.loop_refine.*` hyperparameters, the loop's tensors and its graph;
	- `src/models/lfm-base.cpp`, `src/models/models.h`: optional low-rank deltas on the short-convolution projections. Other models pass none, so their graphs are unchanged;
	- `src/llama-arch.h`, `src/llama-arch.cpp`: one new tensor type, `LLM_TENSOR_LOOP_GATE` (`blk.%d.loop_gate`), appended at the end of the enum so existing values do not move;
	- `src/llama-context.cpp`: the LFM3 loop node budget now also covers models whose routers start at a later layer.

	Applying both patches to a fresh export of the pinned commit reproduces the validated runtime source exactly: no files differ.

	The same source is published as a branch: [renoirvieir4/llama.cpp, branch `round2-1.7b-loop-preview`](https://github.com/renoirvieir4/llama.cpp/tree/round2-1.7b-loop-preview). It contains these two patches as commits on top of `ff0d396e3`, plus a tested recipe and the script `scripts/round2-loop-preview.sh`. Cloning that branch replaces the `git apply` steps below.

	```bash
	git clone https://github.com/ggml-org/llama.cpp.git
	cd llama.cpp
	git checkout ff0d396e3dda0f33e7ce5ad4267fce4f7690cfb1
	PATCH_DIR=/absolute/path/to/round2-1.7b-loop-preview/runtime
	git apply "$PATCH_DIR/01-lfm3-loop-prerequisites.patch"
	git apply "$PATCH_DIR/02-lfm3-refine-loop.patch"
	```

	## Build and run

	```bash
	cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DGGML_BACKEND_DL=ON -DGGML_NATIVE=OFF
	cmake --build build --config Release --target llama-server -j
	./build/bin/llama-server -m /absolute/path/to/round2-1.7b-loop-preview-BF16.gguf \
	-ngl 99 -fa on -c 65536 --temp 1.0 --top-k 20 --top-p 0.95 --repeat-penalty 1.1
	```

	The tested build:
	- Toolchain: CUDA 12.8 and GCC 14, CUDA architecture 86 (RTX 3090), with `-DGGML_BACKEND_DL=ON -DGGML_OPENMP=OFF`.
	- Why OpenMP was off: that machine's GCC 14 package shipped only a static `libgomp`. With a regular compiler installation, OpenMP can stay on.
	- Result: it served the GGUF and answered a multiplication question correctly at 89 tokens/s.

	Other toolchains may need different options.

	## What the GGUF contains

	The GGUF uses the standard `lfm3` architecture with a 40-layer execution plan:
	- Plan: physical layers 0-20, then 5-20 again, then 21-23.
	- Execution metadata: `lfm3.block_count` is 40, and the per-layer arrays (`attention.head_count_kv`, `attention.sliding_window_pattern`) are in execution order.
	- Loop keys: `lfm3.loop_refine.version` = 1, `begin` = 5, `end` = 21, `passes` = 2, `streams` = 2, `physical_layers` = 24, `delta_rank` = 64.
	- Base tensors: each physical layer's tensors keep their standard names under the execution index of its first visit, so layers 21-23 are stored as `blk.37`-`blk.39`.
	- Extra-pass tensors (execution layers 21-36), all FP32:
	- `blk.N.hc_attn_fn.weight` {4096, 8}, `blk.N.hc_attn_base.bias` {8}, and the same for `hc_ffn`: the two-stream mHC routers;
	- `blk.N.<projection>.delta_down` {n_in, 64} and `blk.N.<projection>.delta_up` {64, n_out}, for `attn_q`, `attn_k`, `attn_v`, `attn_output`, `shortconv.in_proj`, `shortconv.out_proj`, `ffn_gate`, `ffn_up` and `ffn_down`;
	- `blk.36.loop_gate.logits` {2048} and `blk.36.loop_gate.token` {2048}.
	- Numerics: the graph applies BF16 rounding at the same points as the BF16 PyTorch reference: the residual streams, the router's pre, post and mix weights, the stream mean, and the gate blend. Native logits then match the reference as closely as stock LFM3-1.7B's do.

	## LM Studio integration used for validation

	The model was validated in LM Studio with a local runtime named `llama.cpp-linux-x86_64-nvidia-cuda12-avx2@2.66.106`.
	- How it was made: it is a copy of LM Studio's own Linux x86_64 CUDA 12 AVX2 runtime 2.66.0 (llama.cpp `ff0d396e3`), where only `libllama.so` was replaced. Every other LM Studio file was kept as shipped: the ggml libraries, the CUDA libraries, the server and the bindings.
	- The replacement library: it was built from the patched tree with `-DGGML_CUDA=OFF -DGGML_BACKEND_DL=ON -DGGML_NATIVE=OFF -DBUILD_SHARED_LIBS=ON`; ggml comes from LM Studio's own libraries. It was then adjusted as follows:

	```bash
	patchelf --set-soname libllama.so --set-rpath '$ORIGIN' \
	--replace-needed libggml.so.0 libggml_llamacpp.so \
	--replace-needed libggml-base.so.0 libggml-base.so \
	/path/to/copied/runtime/libllama.so
	```
	- Registering it: the copied folder went next to the other runtimes in `~/.lmstudio/extensions/backends/`, with a new version in its `backend-manifest.json`. It was then selected with `lms runtime select llama.cpp-linux-x86_64-nvidia-cuda12-avx2@<version>`.
	- Load settings: full GPU offload, flash attention on, context 65,536, one parallel prediction, and a single GPU. That used about 5.2 GB of VRAM.

	This is a version-specific integration, not a general drop-in for other LM Studio runtimes. Automatic runtime updates may switch away from it.

	## Checks

	- Regressions: the CPU architecture suite (`test-llama-archs`: 175 tests over 162 architectures) gave identical results before and after patch 02. Greedy outputs were also token-for-token identical on models of the architectures patch 01 already supported. See `../validation/regression.json`.
	- Agreement with PyTorch: native logits match the PyTorch reference, and the loop's own effect is reproduced. See `../validation/native-vs-pytorch.json`.