Great on 5090 laptop (24 GB) - Excellent responses in quick tests.

#14
by meckdahl - opened

I loaded and used with Ollama and just in its chat had fantastic responses.

I gave it some Claude file discussion on chat email system and it did spot on analysis and recommendations.
I asked it for improvements and it gave a 6 point recommendation (similar to Claude last year)

Very impressive.

It used about 11 Gb of VRAM for a simple query, I wonder if my 5080 desktop card could run it.
gpu-1b

Thanks for the field report, Mark. The 11 GB figure is the useful anchor. A desktop 5080 carries 16 GB, so the same build fits with about 5 GB to spare for that workload. The margin is eaten by context. The KV cache grows toward the 256K window, and image and audio inputs add to it, so long sessions are where a 16 GB card feels the squeeze first. Keeping the context length modest in Ollama keeps the 5080 comfortable. The six point improvement list is a fair workload for a 12B running locally.

Sign up or log in to comment