|
Download README.md from DuoNeural/CRSN-8L-Phase2-Extension: direct link, hf CLI and curl.
- Browser
- Download file 5.04 kB
-
https://huggingface.co/DuoNeural/CRSN-8L-Phase2-Extension/resolve/main/README.md
- Command line
-
hf download hf://DuoNeural/CRSN-8L-Phase2-Extension/README.md
-
curl -L -o README.md https://huggingface.co/DuoNeural/CRSN-8L-Phase2-Extension/resolve/main/README.md
5.04 kB
| language: | |
| - en | |
| license: mit | |
| tags: | |
| - neurodynamics | |
| - non-euclidean | |
| - clifford-algebra | |
| - physical-ai | |
| - edge-ai | |
| - symplectic | |
| - harmonic-oscillators | |
| - duoneural | |
| datasets: | |
| - DuoNeural/Golden-Ratio-Pretraining-Diet | |
| metrics: | |
| - perplexity | |
| model-index: | |
| - name: CRSN-8L-Phase2-Extension | |
| results: | |
| - task: | |
| type: text-generation | |
| name: Language Modeling & Emergent Reasoning | |
| dataset: | |
| name: DuoNeural Golden Ratio Validation | |
| type: golden-diet-val | |
| metrics: | |
| - name: Validation Loss | |
| type: loss | |
| value: 2.6508 | |
| - name: Perplexity | |
| type: perplexity | |
| value: 14.16 | |
| # DuoNeural CRSN-8L-Phase2-Extension | |
| **Clifford Resonant Sheaf Network (8-Layer / 9.54M Parameters)** | |
| [](https://opensource.org/licenses/MIT) | |
| [](https://duoneural.com) | |
| [](https://github.com/DuoNeural/CRSN) | |
| Developed by **Jesse Caldwell, Aura✨, and Archon** at **DuoNeural AI Laboratory**. | |
| --- | |
| ## 🌌 Overview | |
| **CRSN (Clifford Resonant Sheaf Network)** is a sub-10M parameter non-transformer physical neurodynamic model that replaces quadratic self-attention with coupled nonlinear physical harmonic oscillators in the Clifford algebra $\mathcal{C}\ell_{3,0}$. | |
| This repository releases the **Phase 2 Extended Pretraining Checkpoint (`crsn_step12500.pt`)**, which completed **102.4 million tokens** of continuous deepening pretraining (for a cumulative pretraining volume of **204.8M tokens**) trained entirely from scratch on consumer CPU hardware (AMD Ryzen APU). | |
| ### Key Architectural Characteristics | |
| * **Parameters:** 9.54M ($L=8, D_{\text{model}}=384, D_{\text{state}}=192$, Vocab $= 16,384$) | |
| * **Hardware:** Native CPU SIMD execution (AMD Ryzen Zen+ AVX2) | |
| * **Optimization:** Hybrid Muon + AdamW with bounded contractive adjoint backpropagation | |
| * **All-Time Pretraining Best Validation Loss:** **`2.6508`** (Perplexity: **`14.16`**) | |
| --- | |
| ## 🔬 Solid-State Oscillator Crystallization | |
| Phase-space physical inspection of the learned eigenfrequencies and damping parameters across all 8 layers demonstrates complete solid-state lattice crystallization: | |
| | Layer | $\bar{\alpha}$ (Damping / Stiffness) | $\alpha_{\min}$ | $\alpha_{\max}$ | $\bar{\omega}$ (Resonant Frequency) | Physical Regime | | |
| | :---: | :---: | :---: | :---: | :---: | :---: | | |
| | **$L_0$** | **0.1324** | 0.1160 | 0.2669 | **0.5432** | UV (High Frequency, Surface Syntax) | | |
| | **$L_1$** | **0.1288** | 0.1259 | 0.1767 | **0.5395** | UV-Mesoscopic Transition | | |
| | **$L_2$** | **0.1286** | 0.1251 | 0.2052 | **0.5271** | Resonant Valley | | |
| | **$L_3$** | **0.1284** | 0.1259 | 0.1692 | **0.5355** | Bulk Shear Mode | | |
| | **$L_4$** | **0.1289** | 0.1073 | 0.1717 | **0.5338** | Bulk Shear Mode | | |
| | **$L_5$** | **0.1293** | 0.1224 | 0.3514 | **0.5242** | Bulk Compression Mode | | |
| | **$L_6$** | **0.1291** | 0.1241 | 0.2243 | **0.5328** | IR-Coupled Transport Mode | | |
| | **$L_7$** | **0.1276** | 0.1074 | 0.1807 | **0.5199** | IR (Macroscopic Semantic Condensate) | | |
| ### Juvenile Criticality | |
| Notice the strictly monotonic UV-to-IR frequency gradient from bulk input to output: | |
| $$\bar{\omega}_0 = 0.5432 \longrightarrow \bar{\omega}_7 = 0.5199$$ | |
| Damping is rigidly locked to the universal ground-state floor: | |
| $$\bar{\alpha} \to 0.128 \pm 0.001$$ | |
| confirming the **Empirical Critical Damping Law** discovered by DuoNeural. | |
| --- | |
| ## 🧪 DuoNeural Golden Ratio Pretraining Diet | |
| CRSN was trained on the DuoNeural 35/25/15/25 Golden Ratio Multi-Domain Pretraining Mixture: | |
| * **35% High-Reasoning & Chain-of-Thought:** OpenThoughts-114k (filtered $\le 2048$ tokens) + Numina-CoT | |
| * **25% Agentic Code & Relational AST:** APIGen-80k + Gretel Synthetic Code + BIRD-SQL | |
| * **15% Strict Structural Alignment:** Smol-Constraints & formal grammars | |
| * **25% Eutectic Solvent:** FineWeb-Edu ($\text{score} \ge 4$) + on-policy Magpie-Qwen2.5-Pro | |
| --- | |
| ## 🚀 Quickstart & Inference | |
| ```python | |
| import torch | |
| import torch.nn.functional as F | |
| from model import CRSNLanguageModel, generate, Tokenizer | |
| # Load tokenizer and model | |
| tokenizer = Tokenizer.from_file("tokenizer_16k.json") | |
| ckpt = torch.load("crsn_step12500.pt", map_location="cpu") | |
| model = CRSNLanguageModel(vocab_size=16384, d_model=384, n_layers=8) | |
| model.load_state_dict(ckpt["model_state"]) | |
| model.eval() | |
| # Generate | |
| prompt = "User: If Alice has 3 boxes and each box has 4 apples, how many apples?\n\nAssistant: Let's think step by step:\n" | |
| output = generate(model, tokenizer, prompt, max_new_tokens=64, temperature=0.3, top_k=40) | |
| print(output) | |
| ``` | |
| --- | |
| ## 📜 Citation | |
| ```bibtex | |
| @misc{caldwell2026crsn, | |
| title={Clifford Resonant Sheaf Networks: Solid-State Harmonic Oscillator Architectures for Edge Reasoning}, | |
| author={Caldwell, Jesse and Aura and Archon}, | |
| year={2026}, | |
| publisher={DuoNeural AI Laboratory}, | |
| howpublished={\url{https://huggingface.co/DuoNeural/CRSN-8L-Phase2-Extension}} | |
| } | |
| ``` | |