README / README.md
capybara1337's picture
Present Dokichat-100M and Dokichat-100M-Instruct
d7d2404
|
Raw History Blame Contribute Delete
1.06 kB
---
title: README
emoji: 👀
colorFrom: blue
colorTo: purple
sdk: static
pinned: false
---
# Dokitek
Software and intelligent-systems engineering company based in Toulouse, France, and publisher of clinical software. Here, we share what we learn training small language models from scratch, mostly for fun.
## Models
- **[Dokichat-100M](https://huggingface.co/Dokitek/Dokichat-100M)** (from *doki doki*, the heartbeat in Japanese): a 100M-parameter French language model trained from scratch on a single RTX 4080. On our French benchmark, it matches models three to five times larger, and leads all models under 1B in grammar and vocabulary.
- **[Dokichat-100M-Instruct](https://huggingface.co/Dokitek/Dokichat-100M-Instruct)**: its chat version, fine-tuned on synthetic French conversations.
## Coming soon
- **French synthetic conversations**: the datasets we built to fine-tune it.
Our models are small, learning-oriented, and will happily make facts up.
[dokitek.com](https://dokitek.com) · [LinkedIn](https://www.linkedin.com/company/dokitek-sarl)