Download README.md from DuoNeural/CRSN-8L-Phase2-Extension: direct link, hf CLI and curl.
- Browser
- Download file 5.04 kB
-
https://huggingface.co/DuoNeural/CRSN-8L-Phase2-Extension/resolve/main/README.md
- Command line
-
hf download hf://DuoNeural/CRSN-8L-Phase2-Extension/README.md
-
curl -L -o README.md https://huggingface.co/DuoNeural/CRSN-8L-Phase2-Extension/resolve/main/README.md
language:
- en
license: mit
tags:
- neurodynamics
- non-euclidean
- clifford-algebra
- physical-ai
- edge-ai
- symplectic
- harmonic-oscillators
- duoneural
datasets:
- DuoNeural/Golden-Ratio-Pretraining-Diet
metrics:
- perplexity
model-index:
- name: CRSN-8L-Phase2-Extension
results:
- task:
type: text-generation
name: Language Modeling & Emergent Reasoning
dataset:
name: DuoNeural Golden Ratio Validation
type: golden-diet-val
metrics:
- name: Validation Loss
type: loss
value: 2.6508
- name: Perplexity
type: perplexity
value: 14.16
DuoNeural CRSN-8L-Phase2-Extension
Clifford Resonant Sheaf Network (8-Layer / 9.54M Parameters)
Developed by Jesse Caldwell, Auraβ¨, and Archon at DuoNeural AI Laboratory.
π Overview
CRSN (Clifford Resonant Sheaf Network) is a sub-10M parameter non-transformer physical neurodynamic model that replaces quadratic self-attention with coupled nonlinear physical harmonic oscillators in the Clifford algebra $\mathcal{C}\ell_{3,0}$.
This repository releases the Phase 2 Extended Pretraining Checkpoint (crsn_step12500.pt), which completed 102.4 million tokens of continuous deepening pretraining (for a cumulative pretraining volume of 204.8M tokens) trained entirely from scratch on consumer CPU hardware (AMD Ryzen APU).
Key Architectural Characteristics
- Parameters: 9.54M ($L=8, D_{\text{model}}=384, D_{\text{state}}=192$, Vocab $= 16,384$)
- Hardware: Native CPU SIMD execution (AMD Ryzen Zen+ AVX2)
- Optimization: Hybrid Muon + AdamW with bounded contractive adjoint backpropagation
- All-Time Pretraining Best Validation Loss:
2.6508(Perplexity:14.16)
π¬ Solid-State Oscillator Crystallization
Phase-space physical inspection of the learned eigenfrequencies and damping parameters across all 8 layers demonstrates complete solid-state lattice crystallization:
| Layer | $\bar{\alpha}$ (Damping / Stiffness) | $\alpha_{\min}$ | $\alpha_{\max}$ | $\bar{\omega}$ (Resonant Frequency) | Physical Regime |
|---|---|---|---|---|---|
| $L_0$ | 0.1324 | 0.1160 | 0.2669 | 0.5432 | UV (High Frequency, Surface Syntax) |
| $L_1$ | 0.1288 | 0.1259 | 0.1767 | 0.5395 | UV-Mesoscopic Transition |
| $L_2$ | 0.1286 | 0.1251 | 0.2052 | 0.5271 | Resonant Valley |
| $L_3$ | 0.1284 | 0.1259 | 0.1692 | 0.5355 | Bulk Shear Mode |
| $L_4$ | 0.1289 | 0.1073 | 0.1717 | 0.5338 | Bulk Shear Mode |
| $L_5$ | 0.1293 | 0.1224 | 0.3514 | 0.5242 | Bulk Compression Mode |
| $L_6$ | 0.1291 | 0.1241 | 0.2243 | 0.5328 | IR-Coupled Transport Mode |
| $L_7$ | 0.1276 | 0.1074 | 0.1807 | 0.5199 | IR (Macroscopic Semantic Condensate) |
Juvenile Criticality
Notice the strictly monotonic UV-to-IR frequency gradient from bulk input to output: Damping is rigidly locked to the universal ground-state floor: confirming the Empirical Critical Damping Law discovered by DuoNeural.
π§ͺ DuoNeural Golden Ratio Pretraining Diet
CRSN was trained on the DuoNeural 35/25/15/25 Golden Ratio Multi-Domain Pretraining Mixture:
- 35% High-Reasoning & Chain-of-Thought: OpenThoughts-114k (filtered $\le 2048$ tokens) + Numina-CoT
- 25% Agentic Code & Relational AST: APIGen-80k + Gretel Synthetic Code + BIRD-SQL
- 15% Strict Structural Alignment: Smol-Constraints & formal grammars
- 25% Eutectic Solvent: FineWeb-Edu ($\text{score} \ge 4$) + on-policy Magpie-Qwen2.5-Pro
π Quickstart & Inference
import torch
import torch.nn.functional as F
from model import CRSNLanguageModel, generate, Tokenizer
# Load tokenizer and model
tokenizer = Tokenizer.from_file("tokenizer_16k.json")
ckpt = torch.load("crsn_step12500.pt", map_location="cpu")
model = CRSNLanguageModel(vocab_size=16384, d_model=384, n_layers=8)
model.load_state_dict(ckpt["model_state"])
model.eval()
# Generate
prompt = "User: If Alice has 3 boxes and each box has 4 apples, how many apples?\n\nAssistant: Let's think step by step:\n"
output = generate(model, tokenizer, prompt, max_new_tokens=64, temperature=0.3, top_k=40)
print(output)
π Citation
@misc{caldwell2026crsn,
title={Clifford Resonant Sheaf Networks: Solid-State Harmonic Oscillator Architectures for Edge Reasoning},
author={Caldwell, Jesse and Aura and Archon},
year={2026},
publisher={DuoNeural AI Laboratory},
howpublished={\url{https://huggingface.co/DuoNeural/CRSN-8L-Phase2-Extension}}
}