File size: 3,262 Bytes
c3f6fac | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 | ---
language:
- en
license: mit
library_name: pytorch
tags:
- continual-learning
- memory-augmented
- transformer-alternative
- zero-backpropagation
- associative-memory
- pytorch
metrics:
- perplexity
- accuracy
---
# 🧬 CellularFlow v4: Memory-Augmented Continual Learning LLM
**Author:** [Celcilin C S](https://github.com/celcilin)
**Codebase:** [https://github.com/celcilin/cellularflow](https://github.com/celcilin/cellularflow)
**Hugging Face:** [https://huggingface.co/celcilin/cellularflow-v4](https://huggingface.co/celcilin/cellularflow-v4)
**License:** MIT
CellularFlow is a memory-augmented neural architecture designed as a continual-learning alternative to standard Transformers. By replacing dense Feed-Forward Networks (FFN/MLP) with **Multi-Head Associative DNA Memory Banks** and an **Episodic Memory Slot Buffer**, CellularFlow decouples factual knowledge storage from sequence reasoning.
It achieves **83.9% domain retention** across sequential domains (eliminating catastrophic forgetting) and enables **zero-backprop streaming learning** during inference.
---
## 🌟 Benchmark Results
| Architecture | Parameters | Perplexity | Accuracy | 5-Domain Retention |
| :--- | :--- | :--- | :--- | :--- |
| **GPT-mini (Vanilla Transformer)** | 810K | 8.51 | 36.4% | 61.8% |
| **CellularFlow v4 (Hybrid CMC)** | **379K (2.1× fewer)** | **2.54 (−70.3%)** | **73.7%** | **83.9% (+22.1 pp)** |
---
## 🚀 Quickstart: Loading from Hugging Face
### 1. Install CellularFlow
```bash
pip install git+https://github.com/celcilin/cellularflow.git
```
### 2. Run Inference
```python
import torch
from huggingface_hub import hf_hub_download
from cellularflow import CellularFlowLM, CellularFlowTrainer
REPO_ID = "celcilin/cellularflow-v4"
# Download model weights and tokenizer from Hugging Face
weights_file = hf_hub_download(repo_id=REPO_ID, filename="CMC_BaseModel.pt")
# Instantiate CellularFlow v4
model = CellularFlowLM(
vocab_size=8192,
dim=512,
n_layers=6,
n_heads=8,
n_entries=256,
context_len=256,
use_episodic=True
)
# Load pretrained weights
state_dict = torch.load(weights_file, map_location="cpu")
model.load_state_dict(state_dict)
model.eval()
print("CellularFlow v4 loaded successfully!")
```
---
## 🔬 Continual Learning Regimes
1. **Mode 1 — Live Learn (`inference_write=True`):** Updates DNA memory values on the fly via Exponential Moving Average (EMA) with zero backward pass. Protected by **Spherical Anisotropy Regularization**.
2. **Mode 2 — Selective Fine-Tuning (`set_mode("selective")`):** Freezes ~85% of the backbone and updates only DNA memory banks, eliminating catastrophic forgetting.
3. **Mode 3 — Episodic Fact Injection (`inject_fact()`):** Writes facts into slot-based episodic memory with temporal decay and post-epoch consolidation.
---
## 📖 Citation
```bibtex
@article{celcilin2026cellularflow,
title={CellularFlow: A Memory-Augmented Continual-Learning Architecture Decoupling Associative Memory from Sequence Reasoning},
author={Celcilin C S},
journal={arXiv preprint},
year={2026},
url={https://github.com/celcilin/cellularflow}
}
```
|