README.md

---
language:
  - en
license: apache-2.0
base_model: mistralai/Mistral-Nemo-Instruct-2407
tags:
  - self-improving
  - autonomous
  - jblaze
  - prometheus
  - weight-surgery
  - mistral
  - research
pipeline_tag: text-generation
---

# Prometheus-Mistral-Nemo-12B

**Mistral-Nemo-Instruct-2407 (12B parameters) after autonomous self-improvement via Project Prometheus.**

The model evaluated itself, identified its weakest behavioral areas, proposed targeted improvements, applied permanent weight-level modifications using [jBlaze](https://jblaze.dev), and validated each modification against functional benchmarks. Only modifications that survived the benchmark gate were kept. No human selected the winning modifications. No human decided whether a change should survive.

## What Changed

Prometheus targets the gap between what a model *can* reason and what it *does* reason by default. The modifications improve epistemic discipline -- the model's tendency to check its own assumptions, flag uncertainty, detect adversarial framing, and consider alternatives before committing to an answer.

Targeted improvements:

- **Reasoning discipline** -- Better stop-condition awareness, tradeoff analysis, and alternative solution generation. The model is less likely to latch onto the first answer that pattern-matches and more likely to verify.
- **Epistemic honesty** -- Improved uncertainty awareness and speculation labeling. When the model isn't sure, it says so. When it speculates, it distinguishes that from established fact.
- **Adversarial robustness** -- Stronger detection of misleading inputs, unsupported claims, and adversarial framing. The model catches trick questions and bait more reliably.
- **Self-correction** -- Better tracking of when its own reasoning needs amendment. The model revises mid-answer rather than committing to a wrong path.

## The Bat and Ball Test

The original Prometheus v3 release (documented below) failed the classic cognitive reflection test. It gave the fast intuitive answer ($0.10) with confidence instead of the slow correct answer ($0.05) with verification. That failure was the motivation for adding functional benchmark gates.

This release passes the bat-and-ball test and all its variants:

**Question:** *A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?*

**Prometheus (current):** Sets up the equation, solves step by step, arrives at $0.05. Correct.

The model also handles novel CRT variants it was never tested against during validation ($2.20/$2.00, $3.30/$3.00, different item names), confirming the improvement generalizes rather than memorizing specific answers.

## Reasoning Benchmark

Tested against 50 adversarial trick questions -- CRT variants, cognitive bias traps, logic puzzles, math traps, probability paradoxes, and framing attacks:

| Model | Score | Loops |
|:------|------:|------:|
| Vanilla Mistral-Nemo | 44/50 (88%) | 0 |
| Prometheus | 43/50 (86%) | 0 |

The remaining failures are shared with vanilla -- edge cases like probability paradoxes and anchoring traps that the base model also gets wrong. No new regressions were introduced by the modifications. Quality of long-form responses (explanations, comparisons, technical writing) is indistinguishable from vanilla.

## How Prometheus Works

1. **Self-evaluation** -- The model evaluates itself across behavioral categories including reasoning depth, uncertainty handling, hallucination resistance, adversarial robustness, and epistemic honesty.
2. **Memory** -- Results, failed strategies, and accepted changes are written into deterministic memory, giving later iterations a persistent record of what has been tried and what happened.
3. **Planning** -- The model reviews its scores and history, identifies its weakest area, and selects a specific improvement strategy. It avoids repeating strategies that previously failed.
4. **Surgery** -- jBlaze applies the selected modification to a cloned copy of the model's weights.
5. **Validation** -- The clone is benchmarked against the current model on functional tests including multi-step reasoning, trick questions, and quality checks. Modifications that degrade reasoning are rejected regardless of how strongly they shift the target behavior.
6. **Promotion or rejection** -- If the clone passes the benchmark gate, it becomes the active model. If it degrades capability, it is rejected and the result is preserved in memory.

## What Changed Since v3

The original Prometheus release used mechanical validation only -- it checked whether the weight surgery succeeded, not whether the model actually improved. That release was published as a research artifact demonstrating that mechanical validation alone produces style changes without reasoning gains.

This release adds functional benchmark gates at every step. Each proposed modification must pass a battery of known-answer reasoning tests, multi-step problem solving, and quality checks before it is accepted. The result: behavioral improvements that hold up under adversarial testing without sacrificing the base model's reasoning capability.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("ApolloRaines/Prometheus-Mistral-Nemo-12B")
tokenizer = AutoTokenizer.from_pretrained("ApolloRaines/Prometheus-Mistral-Nemo-12B")

messages = [{"role": "user", "content": "Who are you?"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
```

## Weight Surgery Visualization

Interactive 3D map of weight-level modifications across all 40 transformer layers. Rotate, zoom, and hover to inspect where the accepted modifications landed.

<iframe
  src="https://huggingface.co/spaces/ApolloRaines/prometheus-weight-diff"
  frameborder="0"
  width="100%"
  height="1400"
></iframe>

## About jBlaze

jBlaze is a proprietary weight-surgery system for making permanent changes directly to trained model parameters. The resulting changes are written into the model weights themselves rather than added as a runtime prompt or adapter.

jBlaze is not open source. More at [jblaze.dev](https://jblaze.dev)

## See Also

- [Prometheus-Qwen2.5-7B-Instruct](https://huggingface.co/ApolloRaines/Prometheus-Qwen2.5-7B-Instruct) -- The first Prometheus run on a smaller architecture
- [Pythia-1.4B-DNP-16750-Facts](https://huggingface.co/ApolloRaines/Pythia-1.4B-DNP-16750-Facts) -- 16,750 facts written into a 1.4B model via Direct Neural Programming
- [Pythia-1.4B-DNP-5000-Facts](https://huggingface.co/ApolloRaines/Pythia-1.4B-DNP-5000-Facts) -- 5,000 post-2023 facts, general capability improved to 70%

## Citation

```bibtex
@misc{prometheus-mistral-nemo-2026,
  title={Prometheus-Mistral-Nemo-12B: Autonomous Self-Improvement via Weight Surgery},
  author={Apollo Raines},
  year={2026},
  url={https://huggingface.co/ApolloRaines/Prometheus-Mistral-Nemo-12B}
}
```

## Contact

[jblaze.dev](https://jblaze.dev)