field_notes.md

# Field Notes: Building Step-Zero (Cognitive Pacemaker)

Step-Zero is a local-first "Cognitive Pacemaker" application designed to break down overwhelming goals into atomic physical actions to overcome executive dysfunction.

Running small language models (≤4B parameters) entirely on consumer hardware forces you to confront the "Alignment Tax" head-on. Here is the honest developer log of the design decisions, engineering failures, and breakthroughs we encountered.

---

## 1. System Architecture

Step-Zero uses a **dual-model orchestrator** executing locally on CPU/laptop hardware via the `llama.cpp` runtime.

```
                  ┌──────────────────────┐
                  │ Goal & UI Input      │
                  └──────────┬───────────┘
                             │
                             ▼
               ┌───────────────────────────┐
               │ Primary Model (Nemotron)  │
               └─────────────┬─────────────┘
                             │
                     [Output Valid?]
                     /             \
                   Yes              No (or "Too Hard")
                   /                 \
                  ▼                   ▼
      ┌──────────────────────┐   ┌───────────────────────┐
      │ Render Action in UI  │   │ Fallback (MiniCPM)    │
      └──────────────────────┘   └───────────────────────┘
```

1. **Primary Model (Nemotron-Mini-4B):** Fine-tuned using Unsloth SFT specifically on daily goal breakdown datasets (`verbs.jsonl`) to produce atomic, physical next-step instructions.
2. **Fallback Model (MiniCPM-3-4B):** An instruction-following model used for stylistic tone adjustments (Calm vs. Encouraging), semantic repetition loops, and simplifying tasks when a user signals "Too Hard".

---

## 2. Technical Alignments & The GBNF Stop-Token Deadlock

During testing under real model inference, we noticed that Nemotron-Mini-4B suffered from severe over-generation, outputting long paragraphs containing prompt template headers, formatting guidelines, and repeat sentences. Every single task triggered our validation filters and fell back to MiniCPM, causing a 25-second latency penalty.

Here are the two breakthroughs that solved this:

### Breakthrough A: Resolving the GBNF Sampler Deadlock
We originally constrained the model using a GBNF grammar:
```gbnf
root ::= [a-zA-Z0-9 ,.!?'-]+
```
We also specified a stop token of `\n` in the inference call (`stop=["\n"]`) to terminate the action after the first sentence. 

**The Failure:** The GBNF grammar restricted the allowed character set to letters, numbers, and basic punctuation. Crucially, it did *not* include the newline character `\n`. Because GBNF constraints are enforced at the sampler level, the model's probability of selecting `\n` was set to exactly zero. 

As a result, the model was mathematically forbidden from generating the stop token. It kept generating sentence after sentence until it hit the hard limit of `max_tokens=64`, leading to severe paragraph bloating and prompt echoing.

**The Fix:** We updated the GBNF grammar to permit newline characters:
```gbnf
root ::= [a-zA-Z0-9 ,.!?\n'-]+
```
This allowed the model to output a newline character as soon as it finished the task, instantly triggering the `stop=["\n"]` check and terminating generation after the first sentence. The model now outputs exactly 6 to 8 words, saving massive CPU latency.

### Breakthrough B: Token-Level Early Stopping for Prompt Echoing
Small models are highly susceptible to "prompt echoing" (repeating headers like `Completed Tasks:` or `Goal:` when prompted). Rather than waiting for the model to generate a full 64-token sequence of echoed prompt template text before filtering it out in Python, we added the headers directly to the llama.cpp stop-words:
```python
stop=["\n", "<extra_id_1>", "Completed Tasks", "Goal:", "User:", "Assistant:"]
```
If the model begins echoing the prompt template, it hits a stop word within 1 or 2 tokens and halts generation immediately. This terminates failed inference loops in under 50ms, allowing an instantaneous fallback to MiniCPM.

---

## 3. Custom UI/UX: The "Fog of War"

To support the cognitive design of the pacemaker (which aims to reduce cognitive load), the app features a minimal, custom dark theme. By leveraging Gradio's CSS custom properties on `:root` and `.dark`, we overrode Gradio's standard elements to deliver a clean, borderless dark UI styled like a game interface:
- **Fog of War:** Completed history is styled with low opacity (`opacity: 0.3`) and blurred (`filter: blur(2px)`) to keep the user's attention anchored on the single next step.
- **Start Button Theme:** Overrode Gradio's default primary orange buttons to match the deep forest green theme of the interface.
- **Timer and Recovery:** Included a clean "Start Over" action flow to allow user recovery once the circuit breaker fires.

---

## 4. Key Takeaways & Small Model Constraints

1. **Syntax Constraints are Sampler Constraints:** Constraining text syntax with GBNF without matching stop criteria can lead to sampler deadlocks. Always align your grammar's allowed character space with your engine's stop sequences.
2. **LoRA Fine-tuning is Highly Sensitive:** A 4B parameter model is highly sensitive to format drift. Minor discrepancies between fine-tuning prompt structure and inference structure will cause formatting breakdowns.
3. **Optimized CPU Execution:** With optimized GGUF quantizations and proper early stopping, both models run under 50ms per token on consumer CPU laptop hardware, making fully offline cognitive assistance viable.