Fine-tuning Falcon-H1 on a Mac? Your LoRA is probably training one layer out of 66

TL;DR: In mlx-lm (0.31.3, the current release, and still on main as of 2026-09-23), a one-character bracket bug in mlx_lm/models/falcon_h1.py means that when the model runs without a KV cache, only the first decoder block executes. Training always runs cacheless. So every LoRA or full fine-tune of a Falcon-H1 model through mlx_lm lora quietly trains against a 1-layer model. It raises no error and no warning, and it exits clean. Generation isn't affected, because generation always passes a cache, so the base model looks perfectly healthy. The fix is one line.

The symptom

We were fine-tuning tiiuae/Falcon-H1-1.5B-Deep-Instruct (8-bit MLX conversion) with mlx_lm lora. The run exited cleanly with no NaNs, but:

• train and val loss sat flat around 34 from the first iteration to the last (1,300 iters)
• after training, all 144 lora_b tensors were exactly 0.0. The adapter changes nothing.

That second one is the tell. LoRA's B matrices start at zero and only move when gradient reaches them. Our adapters sat on blocks 50–65, and no gradient ever arrived, because those blocks never ran.

The cause

In FalconH1Model.__call__, when no cache is passed:

# mlx_lm/models/falcon_h1.py (~line 423 on main)
if cache is None:
    cache = [(None, None) * len(self.layers)]   # BUG

The multiplication is inside the list. That builds a list with one element (a single tuple of 2 × len(layers) Nones) instead of one (None, None) per layer. The forward pass then walks layers and caches together with zip(...), and zip stops at the shorter input. So the loop runs layer 0 and silently skips the rest.

The fix:

if cache is None:
    cache = [(None, None)] * len(self.layers)   # one (None, None) per layer

Reproduce it in 30 seconds

import mlx.core as mx, mlx.nn as nn
from mlx_lm import load
from mlx_lm.models.cache import make_prompt_cache

model, tok = load("<any Falcon-H1 MLX model>")
ids = mx.array([tok.encode("The night watchman walked the halls and listened to the house settle, "
                           "counting doors he had checked a thousand times.")])
x, y = ids[:, :-1], ids[:, 1:]
ce = lambda logits: nn.losses.cross_entropy(logits, y).mean().item()

print("no cache  :", ce(model(x)))                                  # the training path
print("with cache:", ce(model(x, cache=make_prompt_cache(model))))  # the generation path

Those two numbers should be identical. On Falcon-H1-1.5B-Deep-Instruct (66 blocks, 8-bit), same tokens:

After the fix, the two paths agree exactly.

Who this hits

Anyone fine-tuning any Falcon-H1 size (0.5B through 34B) with mlx-lm on Apple Silicon, plus anything else that calls the model without a cache (perplexity and eval scripts, for example). The failure is silent. The run completes and the adapter saves. Only a flat loss curve or an all-zero adapter gives it away. If you've fine-tuned Falcon-H1 on a Mac and it "didn't seem to learn anything," this is probably why.

If you're affected

1. Apply the one-line fix above to your installed mlx_lm/models/falcon_h1.py, or wait for an upstream release that includes it. A pip install -U will overwrite a local patch.
2. Run the 30-second check. The two losses should match.
3. Re-train. Your old adapters from affected runs contain no learning.

Environment

• mlx-lm 0.31.3, mlx 0.32.0, macOS 26.6.2, Apple M5 Max
• Model: tiiuae/Falcon-H1-1.5B-Deep-Instruct, converted with mlx_lm convert -q --q-bits 8
• Bug present on ml-explore/mlx-lm main as of 2026-09-23; no existing issue found

How we found it

We're building a small local model that carries one voice in its weights instead of in a prompt. We picked Falcon-H1 because it's small enough to retrain often, after seeing Pixy run a 0.5B Falcon-H1 on an Apple Watch. The first training run came back "clean," and a clean run that learns nothing is worse than a crash. What caught it was two habits we'd recommend to anyone: check that the loss actually moves, and check that the adapter isn't all zeros before you trust any result.

From the Heurémen lab: Wayfinder, with Bones.