File size: 3,367 Bytes
743716f dfd0d71 743716f dfd0d71 743716f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 | # SPM-350M
**SPM-350M** is a 350 million parameter decoder-only language model developed by **Syclient**.
The model is designed for research, education, conversational AI, and efficient language model development. SPM-350M serves as the foundation of the Strategic Pretrained Model (SPM) family and represents Syclient's first large-scale language model project.
---
## Overview
SPM-350M is part of the SPM (Strategic Pretrained Model) family, a research initiative focused on building efficient and scalable language models from the ground up.
The project explores language modeling, instruction following, conversational AI, tokenizer design, and future experimental architectures.
---
## Model Specifications
| Property | Value |
| --------------------------- | ------------------------ |
| Model Name | SPM-350M |
| Organization | Syclient |
| Architecture | Decoder-only Transformer |
| Parameters | 348.7M |
| Hidden Size | 1024 |
| Layers | 24 |
| Attention Heads | 16 |
| KV Heads | 8 |
| Context Length | 2048 |
| Recommended Sequence Length | 1024 |
| Vocabulary Size | 32,000 |
| Primary Language | English |
| Secondary Language | Turkish |
---
## Training Data
SPM-350M was trained using a mixture of publicly available datasets, including:
* FineWeb-Edu
* Wikipedia
* UltraChat
* OpenAssistant
The training corpus was curated for research, conversational modeling, and general language understanding tasks.
---
## Model Weights
The model weights are available on Hugging Face.
Coming Soon.
## Intended Use
SPM-350M is intended for:
* Research
* Education
* Conversational AI
* Text Generation
* Experimentation
* Language Model Development
---
## Limitations
SPM-350M is an experimental research model.
The model may:
* Generate incorrect information
* Produce biased outputs
* Make reasoning mistakes
* Hallucinate facts
* Fail on complex tasks
Users should independently verify important information.
---
## Roadmap
* [x] SPM-70M Vertigo
* [x] SPM-350M Foundation
* [x] SPM-350M Chat Edition
* [ ] SPM-350M Ultra Edition
* [ ] SPM-1.2B
* [ ] SPM Flux Architecture
* [ ] SPM-8B
---
## Vision
Our goal is to build efficient language models that are accessible, practical, and scalable.
Rather than focusing solely on model size, the SPM project explores how far efficiency, data quality, and architecture design can be pushed within limited computational resources.
---
## License
Apache License 2.0
---
## Citation
```bibtex
@misc{spm350m2026,
title={SPM-350M},
author={Syclient},
year={2026},
publisher={GitHub}
}
```
---
**Building efficient language models from the ground up.**
|