--- title: README emoji: ๐Ÿ‘€ colorFrom: blue colorTo: purple sdk: static pinned: false --- # Dokitek Software and intelligent-systems engineering company based in Toulouse, France, and publisher of clinical software. Here, we share what we learn training small language models from scratch, mostly for fun. ## Models - **[Dokichat-100M](https://huggingface.co/Dokitek/Dokichat-100M)** (from *doki doki*, the heartbeat in Japanese): a 100M-parameter French language model trained from scratch on a single RTX 4080. On our French benchmark, it matches models three to five times larger, and leads all models under 1B in grammar and vocabulary. - **[Dokichat-100M-Instruct](https://huggingface.co/Dokitek/Dokichat-100M-Instruct)**: its chat version, fine-tuned on synthetic French conversations. ## Coming soon - **French synthetic conversations**: the datasets we built to fine-tune it. Our models are small, learning-oriented, and will happily make facts up. [dokitek.com](https://dokitek.com) ยท [LinkedIn](https://www.linkedin.com/company/dokitek-sarl)