Text Generation
MLX
Safetensors
English
gemma4
ai
artificial-intelligence
machine-learning
ml
post-training
reasoning
local-llm
edge-ai
language-model
apple-silicon
gemma
gemma-4
conversational
4-bit precision
Instructions to use luisbrel/jimmy-4b-zero-regression with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use luisbrel/jimmy-4b-zero-regression with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("luisbrel/jimmy-4b-zero-regression") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use luisbrel/jimmy-4b-zero-regression with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "luisbrel/jimmy-4b-zero-regression"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "luisbrel/jimmy-4b-zero-regression" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use luisbrel/jimmy-4b-zero-regression with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "luisbrel/jimmy-4b-zero-regression"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "luisbrel/jimmy-4b-zero-regression" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "luisbrel/jimmy-4b-zero-regression", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use luisbrel/jimmy-4b-zero-regression with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "luisbrel/jimmy-4b-zero-regression"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default luisbrel/jimmy-4b-zero-regression
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use luisbrel/jimmy-4b-zero-regression with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "luisbrel/jimmy-4b-zero-regression"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "luisbrel/jimmy-4b-zero-regression" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
docs(audit): add G.132/G.142 sample-size disclosure (4 unique prompts x 2 trials) and remove legacy ungrounded draft files
Browse files- LINKEDIN_DRAFT_4B_CAPACITY_CEILING.md +0 -45
- README.md +26 -28
- linkedin_post_clean.txt +0 -34
- linkedin_post_visual.txt +0 -25
LINKEDIN_DRAFT_4B_CAPACITY_CEILING.md
DELETED
|
@@ -1,45 +0,0 @@
|
|
| 1 |
-
# LinkedIn Launch & Research Post: The 4B Parameter Capacity Ceiling
|
| 2 |
-
|
| 3 |
-
**Model Checkpoint**: `luisbrel/jimmy-4b-zero-regression`
|
| 4 |
-
**Target Platform**: LinkedIn
|
| 5 |
-
**Theme**: Empirical testing of the 4B edge parameter ceiling, zero-regression representation surgery, and community invitation
|
| 6 |
-
**IP Protocol**: High-level empirical framing · No internal mathematical recipes or proprietary code disclosed
|
| 7 |
-
|
| 8 |
-
---
|
| 9 |
-
|
| 10 |
-
```text
|
| 11 |
-
Project Jimmy: Closing out our 4B parameter research on Apple Silicon.
|
| 12 |
-
|
| 13 |
-
Can post-training weight surgery enable a compact edge model to reason like a frontier model, or does physical parameter volume impose a hard limit?
|
| 14 |
-
|
| 15 |
-
Here is the empirical head-to-head across three distinct tiers:
|
| 16 |
-
|
| 17 |
-
1. Base Gemma 4B:
|
| 18 |
-
- GSM8K Math: 90.0%
|
| 19 |
-
- HumanEval Code: 90.0%
|
| 20 |
-
- Visual Critic: 35/100 (acute 180px gap between feet and pedals)
|
| 21 |
-
|
| 22 |
-
2. Jimmy 4B (Our Best Checkpoint):
|
| 23 |
-
- GSM8K Math: 90.0% (0.0% regression)
|
| 24 |
-
- HumanEval Code: 90.0% (0.0% regression)
|
| 25 |
-
- Visual Critic: 75/100 (+40 pts leap; exact pedal-foot lock, hollow rims, amber pouch)
|
| 26 |
-
|
| 27 |
-
3. Frontier Reference (Gemini Flash):
|
| 28 |
-
- GSM8K Math: 95.2%
|
| 29 |
-
- HumanEval Code: 92.4%
|
| 30 |
-
- Visual Critic: 100/100 (publication-grade multi-scale artistry)
|
| 31 |
-
|
| 32 |
-
What this proves:
|
| 33 |
-
Post-training surgery successfully repaired localized relational defects with zero anchor regression. It pushed the 4B model to its physical Pareto frontier.
|
| 34 |
-
|
| 35 |
-
Why we are parking Jimmy:
|
| 36 |
-
A 4B architecture has ~4.3 billion parameters. Holding dozens of simultaneous Bezier curves and volumetric proportions in active memory requires parameter bandwidth that 4B models physically lack.
|
| 37 |
-
|
| 38 |
-
To bridge the gap from 75 to 100, surgery must be applied to larger capacity tiers: 12B, 14B, and MoE.
|
| 39 |
-
|
| 40 |
-
The Jimmy weights and complete 2-trial scorecard are live on Hugging Face (link in first comment). If you test the checkpoint, please leave a comment on the model card with your findings.
|
| 41 |
-
|
| 42 |
-
Next stop: scaling beyond 4B.
|
| 43 |
-
|
| 44 |
-
#LocalLLM #AppleSilicon #MachineLearning
|
| 45 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
README.md
CHANGED
|
@@ -31,7 +31,7 @@ pipeline_tag: text-generation
|
|
| 31 |
|
| 32 |
**luisbrel/jimmy-4b-zero-regression** is an experimental compact language model developed locally on Apple Silicon M4 Max using the MLX framework.
|
| 33 |
|
| 34 |
-
This project investigates whether a compact edge architecture can undergo targeted
|
| 35 |
|
| 36 |
---
|
| 37 |
|
|
@@ -41,7 +41,7 @@ Direct comparison across model tiers—from dense edge baseline to our calibrate
|
|
| 41 |
|
| 42 |

|
| 43 |
|
| 44 |
-
| Base Gemma-4 (4B) |
|
| 45 |
| :---: | :---: | :---: |
|
| 46 |
|  |  |  |
|
| 47 |
| **Score: 35 / 100** | **Score: 75 / 100** | **Score: 100 / 100** |
|
|
@@ -49,44 +49,42 @@ Direct comparison across model tiers—from dense edge baseline to our calibrate
|
|
| 49 |
|
| 50 |
### Quantitative Comparison Matrix
|
| 51 |
|
| 52 |
-
| Capability / Benchmark Suite | Base Gemma-4 (4B) |
|
| 53 |
| :--- | :---: | :---: | :---: | :--- |
|
| 54 |
| **Model Scale & Type** | 4.3B Edge Dense | 4.3B Edge Dense (Calibrated) | Frontier Cloud Scale | Compact edge vs calibrated edge vs frontier |
|
| 55 |
-
| **GSM8K Math (
|
| 56 |
-
| **HumanEval Code (
|
| 57 |
-
| **IFEval (Strict Constraints)** | 100.0% |
|
| 58 |
-
| **ARC-Challenge (
|
| 59 |
-
| **Pelican
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
| **Facial Anatomy & Eye (25 pts)** | 10 / 25 (Missing pupil) | **0 / 25** (Pupil omitted in plan) | **25 / 25** (Articulate pupil & pouch) | 4B trade-off: attention budget focused on pedals |
|
| 63 |
-
| **4D Visual Rigor Total Score** | **35 / 100** | **75 / 100** (+40 pts) | **100 / 100** (Reference) | 4B Pareto limit achieved; 100 requires 12B+ |
|
| 64 |
|
| 65 |
---
|
| 66 |
|
| 67 |
-
##
|
| 68 |
|
| 69 |
-
Evaluated
|
| 70 |
|
| 71 |
-
| Benchmark Suite | Base Gemma-4
|
| 72 |
-
| :--- | :---: | :---: | :---
|
| 73 |
-
| **GSM8K** (
|
| 74 |
-
| **MATH-500** (
|
| 75 |
-
| **HumanEval** (
|
| 76 |
-
| **IFEval** (
|
| 77 |
-
| **ARC-Challenge** (
|
| 78 |
-
| **Artificial Analysis v5** (
|
| 79 |
-
| **GUI & DOM Grounding** (
|
| 80 |
-
| **MACRO AVERAGE** |
|
| 81 |
|
| 82 |
---
|
| 83 |
|
| 84 |
-
## 🔬 Key Empirical
|
| 85 |
|
| 86 |
-
In our
|
| 87 |
|
| 88 |
-
* **
|
| 89 |
-
|
| 90 |
* **The Physical Capacity Ceiling**:
|
| 91 |
While localized geometric bugs can be resolved without anchor degradation, compact 4B models hit an undeniable information-theoretic limit: holding 50+ simultaneous multi-scale Bezier curves and organic volumetric proportions in memory requires parameter bandwidth and active attention head count that 4B architectures physically do not possess.
|
| 92 |
* **The Takeaway**:
|
|
|
|
| 31 |
|
| 32 |
**luisbrel/jimmy-4b-zero-regression** is an experimental compact language model developed locally on Apple Silicon M4 Max using the MLX framework.
|
| 33 |
|
| 34 |
+
This project investigates whether a compact 4B edge architecture can undergo targeted geometric calibration while preserving core reasoning behavior on small-sample probe suites (`n=56` total autoregressive evaluations: `4` questions × `2` trials across 7 suites, `69.64% -> 78.57%`, `+8.93%` macro in `outputs/jimmy_4b_zero_regression/02_findings_and_evaluations/HEAD_TO_HEAD_SCORECARD.json` @ `1e1b876`, `M4 Max 128GB MLX`; standalone spatial check in `reports/pelican_benchmark_showcase/jimmy_4b_progressive_gen14.json` @ `ba206f1`). Under `ParetoAnchorGate.strict_rsi_gate(min_samples_per_suite=50)` (`G.142`, `reports/PARETO_POWERED_GATE_AUDIT.json` @ `353dfb8`, binomial 95% upper bound `52.71%` at `n=4` vs `5.82%` at `n=50`), this `n=4`/`n=8` pilot is classified **`REJECTED_UNDERPOWERED`**. `[DISPUTED — RE-RUN REQUIRED]` for full-scale zero-regression generalizations beyond this `n=56` pilot.
|
| 35 |
|
| 36 |
---
|
| 37 |
|
|
|
|
| 41 |
|
| 42 |

|
| 43 |
|
| 44 |
+
| Base Gemma-4 (4B) | `jimmy-4b` (Post-Trained 4B) | Gemini Flash (Frontier Reference — Unverified Summary) |
|
| 45 |
| :---: | :---: | :---: |
|
| 46 |
|  |  |  |
|
| 47 |
| **Score: 35 / 100** | **Score: 75 / 100** | **Score: 100 / 100** |
|
|
|
|
| 49 |
|
| 50 |
### Quantitative Comparison Matrix
|
| 51 |
|
| 52 |
+
| Capability / Benchmark Suite | Base Gemma-4 (4B) | `jimmy-4b` (Post-Trained 4B) | Frontier Reference `[DISPUTED — RE-RUN REQUIRED]` | Analysis & Delta |
|
| 53 |
| :--- | :---: | :---: | :---: | :--- |
|
| 54 |
| **Model Scale & Type** | 4.3B Edge Dense | 4.3B Edge Dense (Calibrated) | Frontier Cloud Scale | Compact edge vs calibrated edge vs frontier |
|
| 55 |
+
| **GSM8K Math** (`n=8`, 4q × 2 trials) | `50.0%` | **`75.0%`** | `[DISPUTED — RE-RUN REQUIRED]` (legacy ungrounded Frontier `95.2%` removed) | `+25.0%` (`outputs/jimmy_4b_zero_regression/02_findings_and_evaluations/HEAD_TO_HEAD_SCORECARD.json` @ `1e1b876`, `M4 Max 128GB MLX`) |
|
| 56 |
+
| **HumanEval Code** (`n=8`, 4q × 2 trials) | `75.0%` | `75.0%` | `[DISPUTED — RE-RUN REQUIRED]` (legacy ungrounded Frontier `92.4%` removed) | `0.0%` (`outputs/jimmy_4b_zero_regression/02_findings_and_evaluations/HEAD_TO_HEAD_SCORECARD.json` @ `1e1b876`) |
|
| 57 |
+
| **IFEval (Strict Constraints)** (`n=8`) | `100.0%` | `100.0%` | `[DISPUTED — RE-RUN REQUIRED]` (legacy ungrounded `98.0%` removed) | `0.0%` (`HEAD_TO_HEAD_SCORECARD.json` @ `1e1b876`) |
|
| 58 |
+
| **ARC-Challenge** (`n=8`, 4q × 2 trials) | `75.0%` | **`87.5%`** | `[DISPUTED — RE-RUN REQUIRED]` | `+12.5%` (`HEAD_TO_HEAD_SCORECARD.json` @ `1e1b876`; pelican SVG rigor `55` / `perceptual 80` in `reports/pelican_benchmark_showcase/jimmy_4b_progressive_gen14.json` @ `ba206f1`, `M4 Max 128GB MLX`) |
|
| 59 |
+
| **Pelican SVG Showcase** (`n=1` static gen14) | `[DISPUTED — RE-RUN REQUIRED]` | `structural 100 / perceptual 80 / rigor 55` | — | `reports/pelican_benchmark_showcase/jimmy_4b_progressive_gen14.json` @ `ba206f1` (`visual_critiques: ["Solid Disc Wheels"]`, `M4 Max 128GB MLX`) |
|
| 60 |
+
|
| 61 |
+
> **Honesty Note on Legacy Qualitative 25-pt Subscores:** `[DISPUTED — RE-RUN REQUIRED]` earlier qualitative claims (`75/100`, hollow rims, `100%` retention, `95.2%`/`92.4%`/`98.0%` Frontier columns) are quarantined; the tracked JSON (`reports/pelican_benchmark_showcase/jimmy_4b_progressive_gen14.json` @ `ba206f1`) explicitly records `visual_rigor_score: 55` and `"Solid Disc Wheels"`. Concept DOI: `10.5281/zenodo.24122803`.
|
|
|
|
|
|
|
| 62 |
|
| 63 |
---
|
| 64 |
|
| 65 |
+
## 📊 Local Head-to-Head Evaluation (`n=56` Pilot, 7 Suites × 4 Questions × 2 Trials)
|
| 66 |
|
| 67 |
+
Evaluated locally on Apple Silicon M4 Max (`128GB` Unified Memory, MLX) across 7 suites (`4` questions × `2` trials = `8` evaluations per suite, `56` total in `outputs/jimmy_4b_zero_regression/02_findings_and_evaluations/HEAD_TO_HEAD_SCORECARD.json` @ `1e1b876` and `EVALUATION_SUMMARY.md` @ `1194cc5`). **Underpowered sample warning:** `n=8`/suite (`n=56` total) is a small pilot and must not be extrapolated to large-scale generalization:
|
| 68 |
|
| 69 |
+
| Benchmark Suite | Base Gemma-4 Mean | `jimmy-4b` Mean | Net Delta (Δ) |
|
| 70 |
+
| :--- | :---: | :---: | :--- |
|
| 71 |
+
| **GSM8K** (4q × 2 trials) | `50.0%` | **`75.0%`** | `+25.0%` (`outputs/jimmy_4b_zero_regression/02_findings_and_evaluations/HEAD_TO_HEAD_SCORECARD.json` @ `1e1b876`, `M4 Max 128GB MLX`) |
|
| 72 |
+
| **MATH-500** (4q × 2 trials) | `25.0%` | **`37.5%`** | `+12.5%` (`HEAD_TO_HEAD_SCORECARD.json` @ `1e1b876`) |
|
| 73 |
+
| **HumanEval** (4q × 2 trials) | `75.0%` | `75.0%` | `0.0%` (`HEAD_TO_HEAD_SCORECARD.json` @ `1e1b876`) |
|
| 74 |
+
| **IFEval** (4q × 2 trials) | `100.0%` | `100.0%` | `0.0%` (`HEAD_TO_HEAD_SCORECARD.json` @ `1e1b876`) |
|
| 75 |
+
| **ARC-Challenge** (4q × 2 trials) | `75.0%` | **`87.5%`** | `+12.5%` (`HEAD_TO_HEAD_SCORECARD.json` @ `1e1b876`) |
|
| 76 |
+
| **Artificial Analysis v5** (4q × 2 trials) | `100.0%` | `100.0%` | `0.0%` (`HEAD_TO_HEAD_SCORECARD.json` @ `1e1b876`) |
|
| 77 |
+
| **GUI & DOM Grounding** (4q × 2 trials) | `62.5%` | **`75.0%`** | `+12.5%` (`HEAD_TO_HEAD_SCORECARD.json` @ `1e1b876`) |
|
| 78 |
+
| **MACRO AVERAGE (`n=56` underpowered pilot)** | `69.64%` | **`78.57%`** | **`+8.93%`** (7-suite macro in `outputs/jimmy_4b_zero_regression/02_findings_and_evaluations/HEAD_TO_HEAD_SCORECARD.json` @ `1e1b876` & `EVALUATION_SUMMARY.md` @ `1194cc5`; see `reports/pelican_benchmark_showcase/jimmy_4b_progressive_gen14.json` @ `ba206f1`, `M4 Max 128GB MLX`; legacy `73.2%` table is `[DISPUTED — RE-RUN REQUIRED]`) |
|
| 79 |
|
| 80 |
---
|
| 81 |
|
| 82 |
+
## 🔬 Key Empirical Observations & Capacity Constraints
|
| 83 |
|
| 84 |
+
In our vector-geometry case study (`reports/pelican_benchmark_showcase/jimmy_4b_progressive_gen14.json` @ `ba206f1`, `M4 Max 128GB MLX`), compact 4B models show clear trade-offs:
|
| 85 |
|
| 86 |
+
* **Audited Pelican Showcase (`jimmy_4b_progressive_gen14`)**:
|
| 87 |
+
Static single-prompt generation completed in `16.198 s` at `113.1 tok/s` (`1832` completion tokens, `n=1` static check vs base Gemma-4 qualitative baselines) with `structural_score: 100`, `perceptual_score: 80`, and `visual_rigor_score: 55` (critique recorded: `"Solid Disc Wheels"`, in `reports/pelican_benchmark_showcase/jimmy_4b_progressive_gen14.json` @ `ba206f1`, `M4 Max 128GB MLX`). `[DISPUTED — RE-RUN REQUIRED]` unverified prose claiming exact `(380, 480)` contact lock or `100%` core reasoning preservation across larger suites.
|
| 88 |
* **The Physical Capacity Ceiling**:
|
| 89 |
While localized geometric bugs can be resolved without anchor degradation, compact 4B models hit an undeniable information-theoretic limit: holding 50+ simultaneous multi-scale Bezier curves and organic volumetric proportions in memory requires parameter bandwidth and active attention head count that 4B architectures physically do not possess.
|
| 90 |
* **The Takeaway**:
|
linkedin_post_clean.txt
DELETED
|
@@ -1,34 +0,0 @@
|
|
| 1 |
-
Project Jimmy: Closing out our 4B parameter research on Apple Silicon.
|
| 2 |
-
|
| 3 |
-
Can targeted post-training calibration enable a compact edge model to reason like a frontier model, or does physical parameter volume impose a hard limit?
|
| 4 |
-
|
| 5 |
-
Here is the empirical head-to-head across three distinct tiers:
|
| 6 |
-
|
| 7 |
-
1. Base Gemma 4B:
|
| 8 |
-
- GSM8K Math: 90.0%
|
| 9 |
-
- HumanEval Code: 90.0%
|
| 10 |
-
- Visual Critic: 35/100 (acute 180px gap between feet and pedals)
|
| 11 |
-
|
| 12 |
-
2. Jimmy 4B (Our Best Checkpoint):
|
| 13 |
-
- GSM8K Math: 90.0% (0.0% regression)
|
| 14 |
-
- HumanEval Code: 90.0% (0.0% regression)
|
| 15 |
-
- Visual Critic: 75/100 (+40 pts leap; exact pedal-foot lock, hollow rims, amber pouch)
|
| 16 |
-
|
| 17 |
-
3. Frontier Reference (Gemini Flash):
|
| 18 |
-
- GSM8K Math: 95.2%
|
| 19 |
-
- HumanEval Code: 92.4%
|
| 20 |
-
- Visual Critic: 100/100 (publication-grade multi-scale artistry)
|
| 21 |
-
|
| 22 |
-
What this proves:
|
| 23 |
-
Targeted post-training calibration successfully resolved localized relational defects with zero anchor regression. It pushed the 4B model to its physical Pareto frontier.
|
| 24 |
-
|
| 25 |
-
Why we are parking Jimmy:
|
| 26 |
-
A 4B architecture has ~4.3 billion parameters. Holding dozens of simultaneous Bezier curves and volumetric proportions in active memory requires parameter bandwidth that 4B models physically lack.
|
| 27 |
-
|
| 28 |
-
To bridge the gap from 75 to 100, these methods must be applied to larger capacity tiers: 12B, 14B, and MoE.
|
| 29 |
-
|
| 30 |
-
The Jimmy weights and complete 2-trial scorecard are live on Hugging Face (link in first comment). If you test the checkpoint, please leave a comment on the model card with your findings.
|
| 31 |
-
|
| 32 |
-
Next stop: scaling beyond 4B.
|
| 33 |
-
|
| 34 |
-
#LocalLLM #AppleSilicon #MachineLearning
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
linkedin_post_visual.txt
DELETED
|
@@ -1,25 +0,0 @@
|
|
| 1 |
-
Can a compact 4B model learn complex spatial relationships, or is there an immutable physical ceiling on 4 billion parameters?
|
| 2 |
-
|
| 3 |
-
Look at the image below. When prompted to draw a pelican riding a bicycle in SVG, the baseline 4B model suffered an acute relational failure: the bird's feet floated 180 pixels in mid-air above the pedals.
|
| 4 |
-
|
| 5 |
-
I spent the past week testing whether targeted post-training calibration on Apple Silicon could repair this disconnect.
|
| 6 |
-
|
| 7 |
-
The calibrated checkpoint (Jimmy-4B) fixed the relational binding:
|
| 8 |
-
- Feet locked directly onto the pedals.
|
| 9 |
-
- Solid black discs became hollow mechanical rims.
|
| 10 |
-
- Natural plumage and amber pouch colors aligned.
|
| 11 |
-
|
| 12 |
-
Crucially, core math and code capabilities stayed 100% stable with zero regression.
|
| 13 |
-
|
| 14 |
-
Then we compared it against a frontier reference (Gemini Flash).
|
| 15 |
-
|
| 16 |
-
Here is the honest finding:
|
| 17 |
-
Targeted calibration can repair structural defects and push a small model to its physical Pareto frontier. But it cannot manufacture capacity that does not physically exist.
|
| 18 |
-
|
| 19 |
-
A 4B model physically runs out of parameter memory bandwidth. Tracking dozens of simultaneous Bezier curves and volumetric proportions in active attention requires parameter volume that only 12B to 14B+ models possess.
|
| 20 |
-
|
| 21 |
-
To reach publication-grade artistry, the next step is taking these methods beyond 4B.
|
| 22 |
-
|
| 23 |
-
The weights and evaluation scorecard are open on Hugging Face (link in first comment). If you test the model, please leave a comment on the card with your findings.
|
| 24 |
-
|
| 25 |
-
#LocalLLM #AppleSilicon #MachineLearning
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|