Commit History
-
Update README.md
Verified signature90c52b0
-
Update README.md
Verified signatured16df96 -
Update config.json
Verified signature358f9c7 -
Update config.json
Verified signature5d80f72 -
Update config.json
Verified signaturef07714c -
Register classes for AutoClasses support
Verified signature5fe95ca
-
Update config.json
Verified signature7f4994c -
Update config.json
Verified signature650b339 -
Update README.md
Verified signaturec52103b -
Final: Correct weight tying keys and add auto_map support
Verified signaturec7143ee -
Final: Correct weight tying keys and add auto_map support
Verified signaturee103f5c -
Final: Correct weight tying keys and add auto_map support
Verified signature8edb089 -
Final: Correct weight tying keys and add auto_map support
Verified signatureaaaa3ae -
Final: Correct weight tying keys and add auto_map support
Verified signaturea868cab -
Final: Correct weight tying keys and add auto_map support
Verified signature664aeb4 -
Final: Correct weight tying keys and add auto_map support
Verified signatureea2c38b -
Final: Correct weight tying keys and add auto_map support
Verified signature14300ec -
Final: Correct weight tying keys and add auto_map support
Verified signaturefbf3750 -
Final: Correct weight tying keys and add auto_map support
Verified signature9d8369f -
Final: Correct weight tying keys and add auto_map support
Verified signatureeabfadf -
Final: Correct weight tying keys and add auto_map support
Verified signature225bfb6 -
Final: Correct weight tying keys and add auto_map support
Verified signature007f5f1 -
Final: Move custom generate logic to model class while maintaining architecture parity
Verified signatureea14637 -
Final: Move custom generate logic to model class while maintaining architecture parity
Verified signature1fc917e -
Final: Move custom generate logic to model class while maintaining architecture parity
Verified signatureb19bed7 -
Final: Move custom generate logic to model class while maintaining architecture parity
Verified signaturea6b6298 -
Final: Include generate method in both modeling and configuration scripts
Verified signatureee5ce56 -
Fix: Use custom generation logic and preserve exact config architecture
Verified signature9eb8fe2 -
Fix: Restore head_dim to config and align with notebook architecture
Verified signature762777d -
Fix: Add GenerationMixin and tie_weights initialization
Verified signatureca9108d -
Upload custom model weights and configuration
Verified signature687817f -
Update README.md
Verified signatured48421c -
Update README.md
Verified signature949a4d9 -
Update README.md
Verified signatureae44245 -
Create README.md
Verified signature# Wiki-SmartBotLM-Instruct > A 270M parameter decoder-only Small Language Model (SLM) pretrained on English Wikipedia and instruction-tuned for general-purpose conversational AI. --- ## Overview **Wiki-SmartBotLM-Instruct** is a compact autoregressive language model developed entirely from scratch as an independent research project. The model was designed to explore the complete lifecycle of modern language model development—from pretraining through instruction tuning—using open datasets and modern transformer architectures. Unlike models that rely on continued pretraining of existing foundation models, Wiki-SmartBotLM-Instruct was initialized with random weights and trained using a multi-stage pipeline. The primary goals of this project were: * Understand large language model training from first principles. * Build a compact yet capable language model. * Explore modern transformer architectures. * Demonstrate the effectiveness of instruction tuning on a pretrained base model. …d0adbe7 -
Upload custom model weights and configuration
Verified signature64d9201 -
initial commit
Verified signature92aa66f