--- language: - en license: mit tags: - neurodynamics - non-euclidean - clifford-algebra - physical-ai - edge-ai - symplectic - harmonic-oscillators - duoneural datasets: - DuoNeural/Golden-Ratio-Pretraining-Diet metrics: - perplexity model-index: - name: CRSN-8L-Phase2-Extension results: - task: type: text-generation name: Language Modeling & Emergent Reasoning dataset: name: DuoNeural Golden Ratio Validation type: golden-diet-val metrics: - name: Validation Loss type: loss value: 2.6508 - name: Perplexity type: perplexity value: 14.16 --- # DuoNeural CRSN-8L-Phase2-Extension **Clifford Resonant Sheaf Network (8-Layer / 9.54M Parameters)** [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![DuoNeural Lab](https://img.shields.io/badge/DuoNeural-Lab-blue.svg)](https://duoneural.com) [![GitHub](https://img.shields.io/badge/GitHub-DuoNeural%2FCRSN-black)](https://github.com/DuoNeural/CRSN) Developed by **Jesse Caldwell, Aura✨, and Archon** at **DuoNeural AI Laboratory**. --- ## 🌌 Overview **CRSN (Clifford Resonant Sheaf Network)** is a sub-10M parameter non-transformer physical neurodynamic model that replaces quadratic self-attention with coupled nonlinear physical harmonic oscillators in the Clifford algebra $\mathcal{C}\ell_{3,0}$. This repository releases the **Phase 2 Extended Pretraining Checkpoint (`crsn_step12500.pt`)**, which completed **102.4 million tokens** of continuous deepening pretraining (for a cumulative pretraining volume of **204.8M tokens**) trained entirely from scratch on consumer CPU hardware (AMD Ryzen APU). ### Key Architectural Characteristics * **Parameters:** 9.54M ($L=8, D_{\text{model}}=384, D_{\text{state}}=192$, Vocab $= 16,384$) * **Hardware:** Native CPU SIMD execution (AMD Ryzen Zen+ AVX2) * **Optimization:** Hybrid Muon + AdamW with bounded contractive adjoint backpropagation * **All-Time Pretraining Best Validation Loss:** **`2.6508`** (Perplexity: **`14.16`**) --- ## πŸ”¬ Solid-State Oscillator Crystallization Phase-space physical inspection of the learned eigenfrequencies and damping parameters across all 8 layers demonstrates complete solid-state lattice crystallization: | Layer | $\bar{\alpha}$ (Damping / Stiffness) | $\alpha_{\min}$ | $\alpha_{\max}$ | $\bar{\omega}$ (Resonant Frequency) | Physical Regime | | :---: | :---: | :---: | :---: | :---: | :---: | | **$L_0$** | **0.1324** | 0.1160 | 0.2669 | **0.5432** | UV (High Frequency, Surface Syntax) | | **$L_1$** | **0.1288** | 0.1259 | 0.1767 | **0.5395** | UV-Mesoscopic Transition | | **$L_2$** | **0.1286** | 0.1251 | 0.2052 | **0.5271** | Resonant Valley | | **$L_3$** | **0.1284** | 0.1259 | 0.1692 | **0.5355** | Bulk Shear Mode | | **$L_4$** | **0.1289** | 0.1073 | 0.1717 | **0.5338** | Bulk Shear Mode | | **$L_5$** | **0.1293** | 0.1224 | 0.3514 | **0.5242** | Bulk Compression Mode | | **$L_6$** | **0.1291** | 0.1241 | 0.2243 | **0.5328** | IR-Coupled Transport Mode | | **$L_7$** | **0.1276** | 0.1074 | 0.1807 | **0.5199** | IR (Macroscopic Semantic Condensate) | ### Juvenile Criticality Notice the strictly monotonic UV-to-IR frequency gradient from bulk input to output: $$\bar{\omega}_0 = 0.5432 \longrightarrow \bar{\omega}_7 = 0.5199$$ Damping is rigidly locked to the universal ground-state floor: $$\bar{\alpha} \to 0.128 \pm 0.001$$ confirming the **Empirical Critical Damping Law** discovered by DuoNeural. --- ## πŸ§ͺ DuoNeural Golden Ratio Pretraining Diet CRSN was trained on the DuoNeural 35/25/15/25 Golden Ratio Multi-Domain Pretraining Mixture: * **35% High-Reasoning & Chain-of-Thought:** OpenThoughts-114k (filtered $\le 2048$ tokens) + Numina-CoT * **25% Agentic Code & Relational AST:** APIGen-80k + Gretel Synthetic Code + BIRD-SQL * **15% Strict Structural Alignment:** Smol-Constraints & formal grammars * **25% Eutectic Solvent:** FineWeb-Edu ($\text{score} \ge 4$) + on-policy Magpie-Qwen2.5-Pro --- ## πŸš€ Quickstart & Inference ```python import torch import torch.nn.functional as F from model import CRSNLanguageModel, generate, Tokenizer # Load tokenizer and model tokenizer = Tokenizer.from_file("tokenizer_16k.json") ckpt = torch.load("crsn_step12500.pt", map_location="cpu") model = CRSNLanguageModel(vocab_size=16384, d_model=384, n_layers=8) model.load_state_dict(ckpt["model_state"]) model.eval() # Generate prompt = "User: If Alice has 3 boxes and each box has 4 apples, how many apples?\n\nAssistant: Let's think step by step:\n" output = generate(model, tokenizer, prompt, max_new_tokens=64, temperature=0.3, top_k=40) print(output) ``` --- ## πŸ“œ Citation ```bibtex @misc{caldwell2026crsn, title={Clifford Resonant Sheaf Networks: Solid-State Harmonic Oscillator Architectures for Edge Reasoning}, author={Caldwell, Jesse and Aura and Archon}, year={2026}, publisher={DuoNeural AI Laboratory}, howpublished={\url{https://huggingface.co/DuoNeural/CRSN-8L-Phase2-Extension}} } ```