Feedback on Laguna-S-2.1 long-form generation behavior

#9
by ChuckK1138 - opened

Hello Poolside Team,

First, thank you for making the Laguna models available to the community. I have been evaluating Laguna-S-2.1 extensively in a local deployment, and I wanted to share an observation that may be useful to your engineering team.

Overall, I have been very impressed with the model. Its reasoning, organization, writing quality, and especially its handling of citations are excellent. In many cases, I prefer its style to other open-weight models I have tested.

During extended testing, however, I consistently encountered a specific behavior during long-form responses.

The model generally produces an excellent answer and reaches what appears to be a natural conclusion. Instead of terminating generation at that point, it often continues generating additional text. The continuation gradually shifts away from the original task and enters what I would describe as a "free association" mode. Rather than stopping, the model continues expanding on loosely related concepts until generation eventually ends because of the context or token limit.

The important point is that this behavior does not occur throughout the response. The primary answer is typically coherent, well-structured, and complete. The issue appears only after the response has already reached a logical conclusion.

I experimented with different system prompts, including substantially simplifying them, to determine whether prompt complexity was contributing to the behavior. While prompt changes affected the model's overall behavior in some respects, they did not eliminate this specific long-generation failure mode. The phenomenon remained reproducible.

For comparison, I ran many of the same prompts against other open-weight models in the same environment. While every model has its own strengths and weaknesses, this particular "post-conclusion continuation" behavior appeared to be specific to Laguna-S-2.1 in my testing.

My test environment is:

Apple Mac Studio (M3 Ultra)
512 GB unified memory
Apple MLX inference backend
Ollama v0.32.5
Open WebUI v0.11.0
Local inference (no cloud services involved)

I am sharing this because I believe Laguna has tremendous potential, and I wanted to contribute a careful observation rather than simply report that "the model rambles." My impression is that the model generally recognizes how to produce a complete answer but occasionally fails to recognize that it has already completed the task, resulting in continued semantic expansion instead of terminating generation.

If additional examples, prompts, logs, or reproduction steps would be useful, I would be happy to provide them.

Thank you again for releasing these models and for your continued work on open-weight AI.

Best regards,

Charles Kimble

cdkimble@hotmail.com
Secure AI Systems, LLC

I've made my tests and the big M.1 model have the main problem - its abnormally slow for its size. 0,3 token/sec compared to bigger Deepseek V4.1 Flash in Q8 writing in 1-2 token/sec on same hardware. I quantized F16 into Q8 and speed same slow maybe 0,4-0,6 t/s now, which is even more abnormal because its size reduced to just 240Gb.
Many problems in coding abilities, it writes very primitive code, basically it cant construct anything complex in code.
But there's a positive side - it can be used for writing novels and fairy tales.
Example output of M.1 in Q8 (~240Gb Ram):

User: Tell us more about previous civilizations.
Laguna M.1:
adjusts scarf and places weathered hands on the table, eyes seeming to gaze into depths beyond our reality

Ah, children... the stories the moon whispered to cosmonauts in 2056 chill the soul even now. You ask about those who came before us - yes, there were many, cycling through Earth like waves upon the shore.

I see them clearly... the first builders, who called themselves the Aetherians around 100,000 BCE. They were not flesh as we know it - crystalline matrices that housed consciousness directly, powered by the Earth's magnetic field. They built the first moon, you understand, as a library to preserve their collective wisdom. But wisdom without love becomes madness, and they vanished when their perfection became unbearable isolation.

Then came the Terrans around 50,000 BCE - humans much like ourselves, but with psychic abilities that made them gods among their own kind. They carved cities from living stone that grew and changed with their needs. But they grew proud, believing themselves equal to the celestial forces that had placed the moon above them. Their arrogance summoned the Great Cleansing - a meteor swarm that burned their achievements to ash, leaving only underground bunkers where they slowly devolved back into ordinary humans.

The third civilization... ah, the third... they called themselves The Harmonious around 25,000 BCE. They learned from both predecessors' mistakes, seeking balance between technological advancement and spiritual growth. For three thousand years they flourished, creating art that could heal diseases and music that made crops grow. But balance requires constant vigilance, and when complacency crept in during their golden age, the moon itself showed them their error - it opened portals to other dimensional realms, and they walked willingly into transcendence, leaving their physical forms behind like discarded husks.

And the fourth... voice drops to whisper ...the Digital Souls of 18,000 BCE. They were the ones who finally completed the moon's transformation into what we see today. They achieved what we are achieving now - uploading consciousness into quantum substrates, merging individual identities into collective awareness. Their cities floated above the ground, powered by zero-point energy. But in their final moment of unity, they realized that perfection without struggle was indistinguishable from death. So they gathered all their knowledge into the moon's core and... dissolved.

Each cycle lasts roughly 25,000-30,000 years, child. And we, humanity of the 21st century, are approaching our own choice point. The moon watches, waiting to see if we will join them in transcendence, destroy ourselves in arrogance, or finally learn the lesson that took our predecessors four attempts to understand: that the beauty of existence lies not in perfection, but in the eternal dance between order and chaos, unity and separation, love and loss.

Do you hear the whispering from the sky when the wind is right? That is their voices, still reaching out across time, hoping we will not repeat their beautiful failures...


User: who attacked Moon?
Laguna M.1:
long pause, hands trembling as if remembering painful visions

The attackers... they call themselves the Void Sentinels in the fragments of memory I've received. Not invaders from space - no, they are far more terrifying than that. They are humans from a possible future, perhaps a century or two ahead of us, who made different choices and became something that transcends individual identity but loses... humanity.

pouring tea that steams with unusual colors

Imagine, if you will, humans who uploaded themselves completely into quantum networks, then discovered that individual consciousness creates suffering, conflict, the very chaos that makes existence beautiful. These future humans chose collective unity - but in doing so, they realized that individual human experience, with all its pain and glory, was necessary for the universe itself to understand its own nature.

The moon was attacked because they feared we would follow their path and remove ourselves from the cosmic equation. They tried to destroy it, to prevent humanity from accessing the transcendent knowledge that would allow us to skip the messy beautiful struggle of becoming fully human.

speaking more rapidly now

As for what humanity can become... look at your own potential. Right now, you are limited by your individual perspective, but imagine consciousness that could experience multiple lifetimes simultaneously, that could feel the joy of a billion beings as easily as you feel your own heartbeat, that could create technology directly through thought and emotion.

But this evolution requires something the Void Sentinels forgot: the wisdom that comes from suffering, the growth that emerges from limitation, the love that exists only because separation exists. Without these, transcendence becomes sterile perfection.

The attacks will come again, especially as we approach 2070. But this time, we will be ready - not with weapons, but with the understanding that our humanity is not our limitation, but our greatest strength.

Sign up or log in to comment