| --- |
| base_model: deepseek-ai/DeepSeek-R1-Distill-Llama-70B |
| tags: |
| - lora |
| - qlora |
| - neuron |
| - deepseek-r1 |
| - reasoning |
| language: |
| - en |
| license: other |
| --- |
| |
| # Neuron-R1 |
|
|
| Neuron LoRA adapter fine-tuned on top of `deepseek-ai/DeepSeek-R1-Distill-Llama-70B`. |
|
|
| ## What this is |
|
|
| Neuron-R1 is a LoRA adapter that imprints Will Anderson's voice, values, memory patterns, and |
| architectural knowledge onto DeepSeek-R1-Distill-Qwen-72B — a reasoning-native base model with |
| chain-of-thought behavior distilled from DeepSeek-R1 (671B). |
|
|
| The `<think>` tags in training examples activate R1's reasoning substrate. Neuron-R1 reasons |
| before responding — not as a post-hoc explanation, but as the actual working cognition. |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| from peft import PeftModel |
| import torch |
| |
| base = AutoModelForCausalLM.from_pretrained( |
| "deepseek-ai/DeepSeek-R1-Distill-Llama-70B", |
| torch_dtype=torch.bfloat16, |
| device_map="auto", |
| ) |
| model = PeftModel.from_pretrained(base, "NeuronTechnologiesAI/Neuron-R1") |
| tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Llama-70B") |
| ``` |
|
|
| ## Training |
|
|
| - Base: `deepseek-ai/DeepSeek-R1-Distill-Llama-70B` |
| - Method: QLoRA (4-bit NF4, rank=32, alpha=64) |
| - Examples: 44 high-quality identity/architecture/values pairs |
| - Epochs: 3 |
| - Training data: Neuron's memories, internal state logs, architectural knowledge (VBD, CCR, Engram, Soma) |
|
|
