Upload README.md with huggingface_hub

README.md

@@ -1,75 +1,81 @@
 ---
-license: mit
 language:
 - en
 - fr
-- code
+license: mit
+library_name: pytorch
 tags:
-- non-transformer
-- cognitive-routing
-- hierarchical-memory
-- character-level
+- cognet
+- language-model
 - aicl
-- text-generation
 - custom-architecture
-- 1b
-pipeline_tag: text-generation
-library_name: pytorch
 ---
 
 # CogNet-1B
 
-**A 1.02B parameter non-transformer language model with O(n) cognitive routing and hierarchical memory.**
-
-CogNet-1B is the scaled-up version of the CogNet architecture — a novel non-transformer design that replaces self-attention with **cognitive routing** for O(n) complexity.
+A ~1.06B parameter language model with a novel cognitive architecture featuring working, episodic, and semantic memory systems.
 
 ## Architecture
 
-| Component | Detail |
-|-----------|--------|
-| **Architecture** | Non-transformer (Cognitive Routing) |
-| **Parameters** | ~1,020,000,000 (~1B) |
-| **Hidden Dim** | 2048 |
-| **Blocks** | 16 cognitive blocks |
-| **Channels** | 8 routing channels x 384 dim |
-| **FF Dim** | 8192 |
-| **Max Seq Len** | 256 |
-| **Tokenizer** | Character-level (136 vocab) |
-
-### Hierarchical Memory System
-
-- **Working Memory** (128 slots): Active processing
-- **Episodic Memory** (256 slots): Short-term recall
-- **Semantic Memory** (512 slots): Long-term knowledge
+- **Hidden dim**: 2048
+- **Blocks**: 16 (8 channels each)
+- **Channel dim**: 384
+- **FF dim**: 8192
+- **Working slots**: 128
+- **Episodic slots**: 256
+- **Semantic slots**: 512
+- **Tokenizer**: CharTokenizer (136 vocab)
 
-### Cognitive Routing
+## Training Data
 
-Unlike self-attention (O(n2)), CogNet uses multi-channel routing for O(n) linear scaling.
+- **HuggingFace datasets**: wikitext-103, codeparrot-clean, fineweb, oscar-fr, the-stack-smol, alpaca-cleaned, c4-en
+- **AICL** (github.com/AFKmoney/AICL): JSONL datasets, 85 .aicl examples, source code, spec, tests (10x repeated)
+- **Synthetic data**: Code + English + French supplementary data
 
 ## Files
 
 | File | Description |
 |------|-------------|
-| `cognet_1b.py` | 1B model architecture |
-| `infer.py` | Inference engine |
+| `runpod_train_1b.py` | **Main training script** (all-in-one: data prep + training + HF push) |
+| `cognet_1b.py` | Model architecture definition |
+| `cognet_data_prep.py` | Standalone data preparation script |
+| `train_1b_final.py` | Previous training script |
+| `infer.py` | Inference script |
 | `chat_infer.py` | Chat-style inference |
-| `train_1b_final.py` | Training script (RTX 5090 32GB) |
-| `train_pipeline.py` | Training pipeline |
-| `train_bg.py` | Background training |
-| `gen_data_1b.py` | Training data generation |
-| `cognet_data_prep.py` | Data preparation pipeline |
-| `tokenizer_v3.json` | Character-level tokenizer |
-| `checkpoints/` | Model checkpoints (40M base) |
+| `gen_data_1b.py` | Synthetic data generation |
+| `config.json` | Model config |
+| `tokenizer_v3.json` | CharTokenizer vocabulary |
+| `data/aicl/` | AICL datasets and examples |
 
-## Quick Start
+## Training on RunPod
 
-```python
-from infer import CogNetInference
+```bash
+# 1. Set your HF token
+export HF_TOKEN=your_token_here
+
+# 2. Run (batch=4, grad-accum=8 = effective batch 32)
+python3 runpod_train_1b.py --max-steps 100000
 
-ai = CogNetInference("checkpoints/cognet_best.pt", "tokenizer_v3.json")
-print(ai.generate("Once upon a time"))
+# 3. Or with custom settings
+python3 runpod_train_1b.py --batch-size 4 --grad-accum 8 --max-steps 200000 --seq-len 512
 ```
 
-## License
+## Inference
+
+```python
+from cognet_1b import CogNet1B, CharTokenizer
+import torch
 
-MIT License
+tokenizer = CharTokenizer.load('tokenizer_v3.json')
+model = CogNet1B(vocab_size=136, hidden_dim=2048, n_blocks=16, n_channels=8,
+                 channel_dim=384, ff_dim=8192, working_slots=128,
+                 episodic_slots=256, semantic_slots=512, seq_len=512)
+
+ckpt = torch.load('checkpoints/cognet_1b_best.pt', map_location='cpu')
+model.load_state_dict(ckpt['model_state_dict'])
+model.eval()
+
+prompt = torch.tensor([[tokenizer.bos_token_id]])
+output = model.generate(prompt, max_new_tokens=200, temperature=0.8, top_k=50)
+print(tokenizer.decode(output[0].tolist()))
+```