# Linkedin post draft **The Best Embedding Models for Southeast Asian Use (and Open Too!)** Excited to share something we've been building for a while — open embedding models, purpose-built for Southeast Asia. Three models, two checkpoints: - **SEA-LION-E5-Embedding-600M** — fine-tuned on multilingual-E5, 512 token context - **SEA-LION-ModernBERT-300M** — ModernBERT + Gemma 3 tokenizer, 8k context - **SEA-LION-ModernBERT-600M** — ModernBERT + Gemma 3 tokenizer, 8k context Trained on 1–2T tokens from the region. We chose ModernBERT as the base for its strong multilingual architecture, and paired it with the Gemma 3 tokenizer — which handles SEA scripts and languages significantly better than most tokenizers out there. Less token fragmentation, better semantic coverage across Thai, Bahasa, Tagalog, Vietnamese and the rest. On performance: all three sit at the top of SEA-BED — the benchmark specifically designed for Southeast Asian language understanding — comparable to or beating multilingual-E5-large, BGE-M3, Qwen, and text-embedding-ada-002 depending on the language. The 600M ModernBERT in particular punches hard across the board. Why care? Embeddings are how AI searches and retrieves — foundational to RAG, search, any agentic system. Most models treat SEA languages as a byproduct. Get that wrong and you get a memory system with blind spots, whole chunks of information that just don't surface when they should. [data point]. We built these so SEA is the default, not the afterthought — while keeping strong English and Chinese performance too (both very much SEA languages 🙂). Models: [HuggingFace link] Blog: [link] [Acknowledgements]