KRAFTON/Raon-SpeechChat-9B
Audio-to-Audio • 10B • Updated • 2.83k • 49
Cahlen Humphreys PRO
cahlen
AI & ML interests
☠️💻
Recent Activity
liked a model 1 day ago
KRAFTON/Raon-SpeechChat-9B upvoted a changelog 1 day ago
Preview LeRobot Episodes liked a model 2 days ago
Edge0/Audio8-ASR-InfiniteOrganizations
liked a
model 1 day ago
upvoted a changelog 1 day ago
Hugging Face Changelog
Preview LeRobot Episodes
• 41
liked a
model 2 days ago
liked 2
models 4 days ago
liked 2
models 9 days ago
liked 2
models 17 days ago
liked 3
models 18 days ago
posted an update 21 days ago
Post
81
I published the serving setup and benchmark results I’ve been using for GLM-5.3-Flash UD-IQ1_S on a single NVIDIA DGX Spark.
The biggest finding was not throughput — it was reliability.
With llama.cpp’s default unrestricted reasoning, a real ~18K-token OpenCode request with 53 tools failed to produce any actionable output in 9/20 runs.
Adding:
--reasoning-budget 2048
changed that to 20/20 successful tool-call responses.
A few other measured results:
* 27.5–29 tok/s decode with MTP vs 18.7 without
* MTP depth 2 outperformed the default depth 3
* 131K context uses ~90.9 GiB resident memory
* 256K context also works on the Spark
* CPU MoE offload does not meaningfully free memory on GB10 unified memory
* newer llama.cpp builds were slightly faster, but introduced tool-call serialization failures, so the repo pins the stable commit
Everything in the README is backed by the benchmark scripts and raw results in the repo.
If you're running GLM-5.3-Flash on a DGX Spark for agentic coding, this should give you a solid starting point.
https://github.com/cahlen/glm-5.3-flash-GGUF-1bit-dgx-spark
unsloth/GLM-5.3-Flash-GGUF
The biggest finding was not throughput — it was reliability.
With llama.cpp’s default unrestricted reasoning, a real ~18K-token OpenCode request with 53 tools failed to produce any actionable output in 9/20 runs.
Adding:
--reasoning-budget 2048
changed that to 20/20 successful tool-call responses.
A few other measured results:
* 27.5–29 tok/s decode with MTP vs 18.7 without
* MTP depth 2 outperformed the default depth 3
* 131K context uses ~90.9 GiB resident memory
* 256K context also works on the Spark
* CPU MoE offload does not meaningfully free memory on GB10 unified memory
* newer llama.cpp builds were slightly faster, but introduced tool-call serialization failures, so the repo pins the stable commit
Everything in the README is backed by the benchmark scripts and raw results in the repo.
If you're running GLM-5.3-Flash on a DGX Spark for agentic coding, this should give you a solid starting point.
https://github.com/cahlen/glm-5.3-flash-GGUF-1bit-dgx-spark
unsloth/GLM-5.3-Flash-GGUF
- 1 reply
liked 4
models 25 days ago
unsloth/Qwen3.8-Flash-Next-GGUF
Image-Text-to-Text • 177B • Updated • 1.76M • 1.07k
zai-org/GLM-5.3
Text Generation • 753B • Updated • 1.16M • • 1.92k
zai-org/GLM-5.3-Flash
Image-Text-to-Text • 321B • Updated • 4.11M • • 2.57k
Qwen/Qwen3.8-Flash-Next
Image-Text-to-Text • 180B • Updated • 847k • • 5.7k
liked 3
models about 1 month ago
