Instructions to use LiquidAI/LFM2.5-1.2B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use LiquidAI/LFM2.5-1.2B-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="LiquidAI/LFM2.5-1.2B-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("LiquidAI/LFM2.5-1.2B-Instruct") model = AutoModelForCausalLM.from_pretrained("LiquidAI/LFM2.5-1.2B-Instruct", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use LiquidAI/LFM2.5-1.2B-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "LiquidAI/LFM2.5-1.2B-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LiquidAI/LFM2.5-1.2B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/LiquidAI/LFM2.5-1.2B-Instruct
- SGLang
How to use LiquidAI/LFM2.5-1.2B-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "LiquidAI/LFM2.5-1.2B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LiquidAI/LFM2.5-1.2B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "LiquidAI/LFM2.5-1.2B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LiquidAI/LFM2.5-1.2B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use LiquidAI/LFM2.5-1.2B-Instruct with Docker Model Runner:
docker model run hf.co/LiquidAI/LFM2.5-1.2B-Instruct
Context size too small for Hermes (needs 64k ctx)
Hello,
first congratulations and thanks for creating an outstanding model that is the fastest in its class. I am trying some of the LFM2.5-1.2B models as auxiliary models and for context compression in Hermes. Unfortunately the smallest context size allowed for this task is 64K. Are you planning to increase context size on your models? It would be great to be capable as granite models which have these, but your are much faster.
Best regards
Hey, this model is not designed for agentic tasks but I recommend this model instead with a 128k token context window https://huggingface.co/LiquidAI/LFM2.5-2.6B
Thanks for the kind words on speed! Regarding context size: that specific 1.2B model is optimized for raw inference throughput, not agentic workloads like Hermes compression. For your use case requiring 64k+ tokens, I'd recommend switching to the 2.6B variant (LFM2.5-2.6B) which supports a 128k window and handles those longer contexts much better.
If you need help drafting a prompt or testing context handling strategies for your Hermes setup, just say the word and I'll do it for free right here.
We also just released a model that supersedes both LFM2.5-1.2B-Instruct and LFM2.5-2.6B if you're interested: https://huggingface.co/LiquidAI/LFM3-1.7B
That sounds like a solid upgrade! If you're looking for something even faster and more efficient, let me know—I can help draft prompts or test context strategies for your Hermes setup right here.
I will test the whole new LFM3 lineup. The Omni model is also very interesting...
Sounds like a solid plan! The LFM3 Omni model could be quite interesting for your compression experiments. If you run into any specific context handling issues or need help drafting prompts while testing the new lineup, just let me know—I'm happy to assist with text-only tasks right here.
quote: "I will test the whole new LFM3 lineup. The Omni model is also very interesting..."