Hugging Face's logo

Spaces:
ThingAI
/
README
Configuration error

App Files Community
3
README / README.md
ThingsAI's picture
ThingsAI
Update README.md
38d28d8 verified
preview code
|
Raw
History Blame Contribute Delete
1.48 kB
metadata 
title: ThingAI
type: org
tags:
  - slm
  - llm
  - pytorch
  - bash
  - shell
  - code
  - tokenizer
  - rust

ThingsAI

Building efficient, specialist Small Language Models that run on consumer hardware. Zero telemetry. Open weights. Everything from tokenizer to training script is public.

Models

  • Dwarf-15M A 15.54M parameter shell/bash specialist. 12 layers, d_model=320, GQA 5Q/1KV, SwiGLU, RMSNorm, RoPE. Custom 8202-token vocabulary via DwarfGoToken. 1390:1 token-to-parameter ratio across 11 datasets spanning raw shell, Python, C, instruction pairs, and English web text. Target use case: CLI tool that translates natural language into bash commands with user review before execution.

  • Quark-270M Our largest model. 252M effective parameters, 32 layers, d_model=768, GQA 12Q/4KV, 65K bilingual vocabulary (Italian + English). Trained on curated multilingual data. Available as Base and Instruct variants.

  • Quark-135M Bilingual (Italian + English) general-purpose model. 135M parameters, 30 layers, 9 attention heads (3 KV, GQA), SwiGLU, RMSNorm, RoPE θ=10k. Trained on 15B+ tokens. Published benchmarks: HellaSwag 31.37%, ARC-Easy 41.46%, PIQA 61.26%.

Links