README / README.md
capybara1337's picture
Present Dokichat-100M and Dokichat-100M-Instruct
d7d2404
|
Raw History Blame Contribute Delete
1.06 kB
metadata
title: README
emoji: 👀
colorFrom: blue
colorTo: purple
sdk: static
pinned: false

Dokitek

Software and intelligent-systems engineering company based in Toulouse, France, and publisher of clinical software. Here, we share what we learn training small language models from scratch, mostly for fun.

Models

  • Dokichat-100M (from doki doki, the heartbeat in Japanese): a 100M-parameter French language model trained from scratch on a single RTX 4080. On our French benchmark, it matches models three to five times larger, and leads all models under 1B in grammar and vocabulary.
  • Dokichat-100M-Instruct: its chat version, fine-tuned on synthetic French conversations.

Coming soon

  • French synthetic conversations: the datasets we built to fine-tune it.

Our models are small, learning-oriented, and will happily make facts up.

dokitek.com · LinkedIn