celcilin commited on
Commit
c3f6fac
Β·
verified Β·
1 Parent(s): e0ca5de

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +97 -0
README.md ADDED
@@ -0,0 +1,97 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: mit
5
+ library_name: pytorch
6
+ tags:
7
+ - continual-learning
8
+ - memory-augmented
9
+ - transformer-alternative
10
+ - zero-backpropagation
11
+ - associative-memory
12
+ - pytorch
13
+ metrics:
14
+ - perplexity
15
+ - accuracy
16
+ ---
17
+
18
+ # 🧬 CellularFlow v4: Memory-Augmented Continual Learning LLM
19
+
20
+ **Author:** [Celcilin C S](https://github.com/celcilin)
21
+ **Codebase:** [https://github.com/celcilin/cellularflow](https://github.com/celcilin/cellularflow)
22
+ **Hugging Face:** [https://huggingface.co/celcilin/cellularflow-v4](https://huggingface.co/celcilin/cellularflow-v4)
23
+ **License:** MIT
24
+
25
+ CellularFlow is a memory-augmented neural architecture designed as a continual-learning alternative to standard Transformers. By replacing dense Feed-Forward Networks (FFN/MLP) with **Multi-Head Associative DNA Memory Banks** and an **Episodic Memory Slot Buffer**, CellularFlow decouples factual knowledge storage from sequence reasoning.
26
+
27
+ It achieves **83.9% domain retention** across sequential domains (eliminating catastrophic forgetting) and enables **zero-backprop streaming learning** during inference.
28
+
29
+ ---
30
+
31
+ ## 🌟 Benchmark Results
32
+
33
+ | Architecture | Parameters | Perplexity | Accuracy | 5-Domain Retention |
34
+ | :--- | :--- | :--- | :--- | :--- |
35
+ | **GPT-mini (Vanilla Transformer)** | 810K | 8.51 | 36.4% | 61.8% |
36
+ | **CellularFlow v4 (Hybrid CMC)** | **379K (2.1Γ— fewer)** | **2.54 (βˆ’70.3%)** | **73.7%** | **83.9% (+22.1 pp)** |
37
+
38
+ ---
39
+
40
+ ## πŸš€ Quickstart: Loading from Hugging Face
41
+
42
+ ### 1. Install CellularFlow
43
+ ```bash
44
+ pip install git+https://github.com/celcilin/cellularflow.git
45
+ ```
46
+
47
+ ### 2. Run Inference
48
+ ```python
49
+ import torch
50
+ from huggingface_hub import hf_hub_download
51
+ from cellularflow import CellularFlowLM, CellularFlowTrainer
52
+
53
+ REPO_ID = "celcilin/cellularflow-v4"
54
+
55
+ # Download model weights and tokenizer from Hugging Face
56
+ weights_file = hf_hub_download(repo_id=REPO_ID, filename="CMC_BaseModel.pt")
57
+
58
+ # Instantiate CellularFlow v4
59
+ model = CellularFlowLM(
60
+ vocab_size=8192,
61
+ dim=512,
62
+ n_layers=6,
63
+ n_heads=8,
64
+ n_entries=256,
65
+ context_len=256,
66
+ use_episodic=True
67
+ )
68
+
69
+ # Load pretrained weights
70
+ state_dict = torch.load(weights_file, map_location="cpu")
71
+ model.load_state_dict(state_dict)
72
+ model.eval()
73
+
74
+ print("CellularFlow v4 loaded successfully!")
75
+ ```
76
+
77
+ ---
78
+
79
+ ## πŸ”¬ Continual Learning Regimes
80
+
81
+ 1. **Mode 1 β€” Live Learn (`inference_write=True`):** Updates DNA memory values on the fly via Exponential Moving Average (EMA) with zero backward pass. Protected by **Spherical Anisotropy Regularization**.
82
+ 2. **Mode 2 β€” Selective Fine-Tuning (`set_mode("selective")`):** Freezes ~85% of the backbone and updates only DNA memory banks, eliminating catastrophic forgetting.
83
+ 3. **Mode 3 β€” Episodic Fact Injection (`inject_fact()`):** Writes facts into slot-based episodic memory with temporal decay and post-epoch consolidation.
84
+
85
+ ---
86
+
87
+ ## πŸ“– Citation
88
+
89
+ ```bibtex
90
+ @article{celcilin2026cellularflow,
91
+ title={CellularFlow: A Memory-Augmented Continual-Learning Architecture Decoupling Associative Memory from Sequence Reasoning},
92
+ author={Celcilin C S},
93
+ journal={arXiv preprint},
94
+ year={2026},
95
+ url={https://github.com/celcilin/cellularflow}
96
+ }
97
+ ```