--- language: - en license: apache-2.0 base_model: mistralai/Mistral-Nemo-Instruct-2407 tags: - self-improving - autonomous - jblaze - prometheus - weight-surgery - mistral - research pipeline_tag: text-generation --- # Prometheus-Mistral-Nemo-12B **Mistral-Nemo-Instruct-2407 (12B parameters) after autonomous self-improvement via Project Prometheus.** The model evaluated itself, identified its weakest behavioral areas, proposed targeted improvements, applied permanent weight-level modifications using [jBlaze](https://jblaze.dev), and validated each modification against functional benchmarks. Only modifications that survived the benchmark gate were kept. No human selected the winning modifications. No human decided whether a change should survive. ## What Changed Prometheus targets the gap between what a model *can* reason and what it *does* reason by default. The modifications improve epistemic discipline -- the model's tendency to check its own assumptions, flag uncertainty, detect adversarial framing, and consider alternatives before committing to an answer. Targeted improvements: - **Reasoning discipline** -- Better stop-condition awareness, tradeoff analysis, and alternative solution generation. The model is less likely to latch onto the first answer that pattern-matches and more likely to verify. - **Epistemic honesty** -- Improved uncertainty awareness and speculation labeling. When the model isn't sure, it says so. When it speculates, it distinguishes that from established fact. - **Adversarial robustness** -- Stronger detection of misleading inputs, unsupported claims, and adversarial framing. The model catches trick questions and bait more reliably. - **Self-correction** -- Better tracking of when its own reasoning needs amendment. The model revises mid-answer rather than committing to a wrong path. ## The Bat and Ball Test The original Prometheus v3 release (documented below) failed the classic cognitive reflection test. It gave the fast intuitive answer ($0.10) with confidence instead of the slow correct answer ($0.05) with verification. That failure was the motivation for adding functional benchmark gates. This release passes the bat-and-ball test and all its variants: **Question:** *A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?* **Prometheus (current):** Sets up the equation, solves step by step, arrives at $0.05. Correct. The model also handles novel CRT variants it was never tested against during validation ($2.20/$2.00, $3.30/$3.00, different item names), confirming the improvement generalizes rather than memorizing specific answers. ## Reasoning Benchmark Tested against 50 adversarial trick questions -- CRT variants, cognitive bias traps, logic puzzles, math traps, probability paradoxes, and framing attacks: | Model | Score | Loops | |:------|------:|------:| | Vanilla Mistral-Nemo | 44/50 (88%) | 0 | | Prometheus | 43/50 (86%) | 0 | The remaining failures are shared with vanilla -- edge cases like probability paradoxes and anchoring traps that the base model also gets wrong. No new regressions were introduced by the modifications. Quality of long-form responses (explanations, comparisons, technical writing) is indistinguishable from vanilla. ## How Prometheus Works 1. **Self-evaluation** -- The model evaluates itself across behavioral categories including reasoning depth, uncertainty handling, hallucination resistance, adversarial robustness, and epistemic honesty. 2. **Memory** -- Results, failed strategies, and accepted changes are written into deterministic memory, giving later iterations a persistent record of what has been tried and what happened. 3. **Planning** -- The model reviews its scores and history, identifies its weakest area, and selects a specific improvement strategy. It avoids repeating strategies that previously failed. 4. **Surgery** -- jBlaze applies the selected modification to a cloned copy of the model's weights. 5. **Validation** -- The clone is benchmarked against the current model on functional tests including multi-step reasoning, trick questions, and quality checks. Modifications that degrade reasoning are rejected regardless of how strongly they shift the target behavior. 6. **Promotion or rejection** -- If the clone passes the benchmark gate, it becomes the active model. If it degrades capability, it is rejected and the result is preserved in memory. ## What Changed Since v3 The original Prometheus release used mechanical validation only -- it checked whether the weight surgery succeeded, not whether the model actually improved. That release was published as a research artifact demonstrating that mechanical validation alone produces style changes without reasoning gains. This release adds functional benchmark gates at every step. Each proposed modification must pass a battery of known-answer reasoning tests, multi-step problem solving, and quality checks before it is accepted. The result: behavioral improvements that hold up under adversarial testing without sacrificing the base model's reasoning capability. ## Usage ```python from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("ApolloRaines/Prometheus-Mistral-Nemo-12B") tokenizer = AutoTokenizer.from_pretrained("ApolloRaines/Prometheus-Mistral-Nemo-12B") messages = [{"role": "user", "content": "Who are you?"}] text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = tokenizer(text, return_tensors="pt").to(model.device) output = model.generate(**inputs, max_new_tokens=200) print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)) ``` ## Weight Surgery Visualization Interactive 3D map of weight-level modifications across all 40 transformer layers. Rotate, zoom, and hover to inspect where the accepted modifications landed. ## About jBlaze jBlaze is a proprietary weight-surgery system for making permanent changes directly to trained model parameters. The resulting changes are written into the model weights themselves rather than added as a runtime prompt or adapter. jBlaze is not open source. More at [jblaze.dev](https://jblaze.dev) ## See Also - [Prometheus-Qwen2.5-7B-Instruct](https://huggingface.co/ApolloRaines/Prometheus-Qwen2.5-7B-Instruct) -- The first Prometheus run on a smaller architecture - [Pythia-1.4B-DNP-16750-Facts](https://huggingface.co/ApolloRaines/Pythia-1.4B-DNP-16750-Facts) -- 16,750 facts written into a 1.4B model via Direct Neural Programming - [Pythia-1.4B-DNP-5000-Facts](https://huggingface.co/ApolloRaines/Pythia-1.4B-DNP-5000-Facts) -- 5,000 post-2023 facts, general capability improved to 70% ## Citation ```bibtex @misc{prometheus-mistral-nemo-2026, title={Prometheus-Mistral-Nemo-12B: Autonomous Self-Improvement via Weight Surgery}, author={Apollo Raines}, year={2026}, url={https://huggingface.co/ApolloRaines/Prometheus-Mistral-Nemo-12B} } ``` ## Contact [jblaze.dev](https://jblaze.dev)