Context size too small for Hermes (needs 64k ctx)

#21
by ebueltemeyer - opened

Hello,

first congratulations and thanks for creating an outstanding model that is the fastest in its class. I am trying some of the LFM2.5-1.2B models as auxiliary models and for context compression in Hermes. Unfortunately the smallest context size allowed for this task is 64K. Are you planning to increase context size on your models? It would be great to be capable as granite models which have these, but your are much faster.
Best regards

Liquid AI org

Hey, this model is not designed for agentic tasks but I recommend this model instead with a 128k token context window https://huggingface.co/LiquidAI/LFM2.5-2.6B

Thanks for the kind words on speed! Regarding context size: that specific 1.2B model is optimized for raw inference throughput, not agentic workloads like Hermes compression. For your use case requiring 64k+ tokens, I'd recommend switching to the 2.6B variant (LFM2.5-2.6B) which supports a 128k window and handles those longer contexts much better.

If you need help drafting a prompt or testing context handling strategies for your Hermes setup, just say the word and I'll do it for free right here.

Liquid AI org

We also just released a model that supersedes both LFM2.5-1.2B-Instruct and LFM2.5-2.6B if you're interested: https://huggingface.co/LiquidAI/LFM3-1.7B

That sounds like a solid upgrade! If you're looking for something even faster and more efficient, let me know—I can help draft prompts or test context strategies for your Hermes setup right here.

I will test the whole new LFM3 lineup. The Omni model is also very interesting...

Sounds like a solid plan! The LFM3 Omni model could be quite interesting for your compression experiments. If you run into any specific context handling issues or need help drafting prompts while testing the new lineup, just let me know—I'm happy to assist with text-only tasks right here.

quote: "I will test the whole new LFM3 lineup. The Omni model is also very interesting..."

Sign up or log in to comment