--- library_name: transformers license: other license_name: lfm1.0 license_link: LICENSE pipeline_tag: text-generation tags: - liquid - lfm3 - edge - conversational - custom_code language: - en - ar - zh - fr - de - it - ja - ko - pt - es - vi - th - id - hi - ru - pl ---

API Try Docs Blog

# LFM3-1.7B-Base LFM3-1.7B-Base is the **pre-trained checkpoint** behind [LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B), a text model for local agents and specialized applications based on our new tri-hybrid architecture. - **Pre-trained on 22T tokens:** the last 3.5T tokens are curated mid-training data with agentic traces and tool calls, and extend the context window to 256K tokens. - **Designed for the edge:** at 4-bit precision, it takes about 1 GB of memory, so it fits on any phone or laptop. - **Made to be specialized:** a starting point for continued pre-training and your own post-training. Find more information about the LFM3 family in [our release blog post](https://www.liquid.ai/blog/lfm3). ## Model details

The [LFM3 family](https://huggingface.co/collections/LiquidAI/lfm3) includes base, text, vision, audio, omni, and decision models.

Property	Specification
Checkpoint	Pre-trained, text-only base model
Parameters	1.73B
Architecture	24 layers: 13 gated short convolutions, 5 sliding-window attention, 6 global attention
Context length	256K tokens (input + output)
Vocabulary	128,000 tokens
Languages	English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish
License	LFM Open License v1.0

LFM3-1.7B-Base is the checkpoint used to create LFM3-1.7B. It is only recommended for tasks that require heavy fine-tuning, like language-specific or domain-specific assistants, training on proprietary data, or experimenting with novel post-training approaches. For chat, tool calling, and agents, use [LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B). ## Inference

LFM3-1.7B-Base uses the same architecture and modeling code as [LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B), with BF16 weights. **Transformers quick start:** ```python from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "LiquidAI/LFM3-1.7B-Base" model = AutoModelForCausalLM.from_pretrained( model_id, device_map="auto", dtype="bfloat16", trust_remote_code=True, ) tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) prompt = "The nematode C. elegans is" input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(model.device) output = model.generate(input_ids, max_new_tokens=128, do_sample=False) print(tokenizer.decode(output[0], skip_special_tokens=True)) ``` Mid-training included chat data with and without thinking traces, formatted with the same chat template as LFM3-1.7B. With `apply_chat_template`, reasoning is enabled by default, so responses start with a thinking trace. Set `enable_thinking=False` to prefill an empty thinking trace and have the model answer directly. ## Fine-tuning

LFM3-1.7B-Base is the starting point for substantial customization, such as adapting to new domains, languages, or behaviors that diverge significantly from those of LFM3-1.7B, through continued pre-training or full post-training via [TRL](https://docs.liquid.ai/lfm/fine-tuning/trl). These notebooks start from LFM3-1.7B. To start from the base model, set `model_id = "LiquidAI/LFM3-1.7B-Base"`.

Training	Use case	Notebook
SFT	Extract structured JSON from messy documents
CPT	Learn a new language (Turkish) from raw text

## Training

**Pre-training:** two stages build LFM3-1.7B-Base on 22T tokens in total. - **Main pre-training:** compared with LFM2, the mix adds more scientific papers, repository-level code, math, and a wider range of knowledge sources. - **Mid-training:** the last 3.5T tokens use curated, higher-quality data with a larger share of agentic traces and tool calls. This stage also extends the context window to 256K tokens. Post-training from this checkpoint (SFT, mixed RL, multi-domain on-policy distillation, and agentic RL) produces [LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B). ## Citation

```bibtex @article{liquidAI2026LFM3, author = {Liquid AI}, title = {Introducing LFM3: Every Modality for Specialized Intelligence}, journal = {Liquid AI Blog}, year = {2026}, note = {www.liquid.ai/blog/lfm3}, } ``` For questions and feedback, join [Discord](https://discord.com/invite/liquid-ai). For custom deployment and fine-tuning support, [contact Liquid AI](https://www.liquid.ai/contact).