LevelBot
🥇
567
Verify your Discord account with Hugging Face login
pavle PRO
pavle-scalably
AI & ML interests
Production AI agents, owned NVIDIA inference, frontier open-weight models, long-context serving, tool use, post-training and evaluation.
Recent Activity
repliedto their post about 1 month ago
14 days serving
https://huggingface.co/unsloth/Qwen3.8-27B-NVFP4 to production agents on 2x RTX 5090 (vLLM 0.27, TP=2, 262K context, FP8 KV):
28,097 requests, 860.6M prompt tokens, 82.6% prefix-cache hit rate, TTFT p50 0.61 s, 0 engine errors.
The observation: prefix cache, not throughput, decides whether a 27B model keeps up with agents.
Mean request is 30,100 tokens in, 983 out, because every turn resends the whole session. Two flags mattered most: --max-num-seqs 12 (queue p95 went 9.4 s to 233.6 s past that) and --watermark 0.08 (preemptions 29 to 2). And thinking off for tool loops: 917 tokens in 11.7 s vs 11,170 in 144 s, same answer.
Full config and counters: scalably.io/blog/qwen3-8-27b-nvfp4-rtx-5090-production
Next we are preparing an 8x B300 node in an EU data center for open-weight serving. Which models or workloads are underserved for you? published an article about 1 month ago
Fast memory, slow weights: how a production agent system improves around the model, and the model itself repliedto dipankarsarkar's post about 1 month ago
Your issue tracker is in the wrong place.
It lives on a server. Your code lives in git. So every time an agent picks up work it makes an API call, burns a token, fights a rate limit, and still cannot see what the other agent just did.
Move the issues into the repo. Append-only event log in git refs. Branches when you branch, merges when you merge, CRDT so two agents never conflict. No server, no database.
The coordination signal that PR-level telemetry misses lives before the pull request. The paper, and a live demo running the real tool:
https://huggingface.co/papers/2606.19616
https://huggingface.co/spaces/neullabs/grite
If your agents share a repo, where does their shared state actually live right now?Organizations
liked 2
Spaces about 1 month ago
