-
Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models
Paper • 2311.06783 • Published • 27 -
The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4
Paper • 2311.07361 • Published • 13 -
GAIA: a benchmark for General AI Assistants
Paper • 2311.12983 • Published • 253 -
teknium/openhermes
Viewer • Updated • 243k • 1.28k • 241
Marcus W
GrihmLord
·
AI & ML interests
AI Bitcoin & Machine Learning
Recent Activity
liked a model about 1 month ago
abenzerps/Qwen-Image-2.1-Uncensored-Quantized liked a model about 1 month ago
medismera/Qwen3.8-27B-OBLITERATED-Mythos-Class-Agentic reacted to FredyRivera-dev's post with ❤️ 3 months ago
We wrote a full technical guide on how to train a bilingual (ES/EN) LLM from scratch: TinyQwen.
Covers:
- Hybrid architecture based on Qwen3.5
- Pre-training with 15B tokens
- Cost benchmark between H200 and B200
- Post-training with SFT + LoRA
- Full code and data, open source
With ~$11 of compute on an H200 we ran an initial training run, enough to validate the full architecture and pipeline.
Blog post: https://aquiles-ai.vercel.app/blog/tinyqwen-from-scratch
Technical feedback welcome, especially from anyone looking to replicate the pipeline with more compute.Organizations
None yet