Hugging Face's logo
ManniX's picture

ManniX PRO

ManniX-ITA
32 6 28
Fernanduli's profile picture Omegakenshin's profile picture Lamsam26's profile picture
·
https://github.com/mann1x

AI & ML interests

None yet

Recent Activity

commentedon a paper about 1 hour ago
Group Entropy-Controlled Policy Optimization
repliedto their post 1 day ago
# opencoti-llamafile 0.10.3-c5 — settled admission for multi-agent serving New cut of the opencoti single-file inference engine (llamafile 0.10.3 / llama.cpp + 87 additive patches). Zero-dependency APE: one executable for Linux, Windows, macOS & BSD. What's new vs c4: **PolyKV fan-out — pool from a live session.** `POST /polykv/pools` gains `from_session`/`from_slot`: the shared prefix is snapshotted server-side from the session's cached KV — no tokens resent, token-exact. Ephemeral pools auto-release when orchestrators die mid-round. **PolyKV P7 — settled admission.** Spawning agents faster than the tps signal settles was oversubscribing pools. Now: a per-pool settle window paces admits just enough for a reliable reading; warming sessions no longer bias the mean; the post-admit forecast uses the measured per-admit drop; idle gaps (agents mid-tool-call) no longer read as free capacity; `guarantee_min_sessions` means a new/nested pool always gets its first agent — capacity checks can never deadlock an orchestrator; the enforced gate applies to new sessions only, with per-request `overcommit`. Benchmark (multi-agent courier, floor 15 tok/s): time-under-floor −52%, deep sub-floor −83%, delivery p50 −43%, 100% task score. **Zero-conf GPU sharing.** Instances on one GPU discover each other over shared memory — no ports, no config — and split compute by `--gpu-share-weight`. Measured (3090): weights 2:1 → 71.7/36.2 tok/s; holds at `--parallel 4` and under MTP. Idle peers cost nothing (solo = full speed), crashes age out in 3 s; `GET /gpu/peers` shows live shares + busy %. From c5 every release ships per-platform side-load DSOs: `dso/<ver>/` with Linux x86_64 + sbsa `.so` and a Windows `.dll`. https://huggingface.co/ManniX-ITA/opencoti-llamafile
repliedto their post 1 day ago
# opencoti-llamafile 0.10.3-c5 — settled admission for multi-agent serving New cut of the opencoti single-file inference engine (llamafile 0.10.3 / llama.cpp + 87 additive patches). Zero-dependency APE: one executable for Linux, Windows, macOS & BSD. What's new vs c4: **PolyKV fan-out — pool from a live session.** `POST /polykv/pools` gains `from_session`/`from_slot`: the shared prefix is snapshotted server-side from the session's cached KV — no tokens resent, token-exact. Ephemeral pools auto-release when orchestrators die mid-round. **PolyKV P7 — settled admission.** Spawning agents faster than the tps signal settles was oversubscribing pools. Now: a per-pool settle window paces admits just enough for a reliable reading; warming sessions no longer bias the mean; the post-admit forecast uses the measured per-admit drop; idle gaps (agents mid-tool-call) no longer read as free capacity; `guarantee_min_sessions` means a new/nested pool always gets its first agent — capacity checks can never deadlock an orchestrator; the enforced gate applies to new sessions only, with per-request `overcommit`. Benchmark (multi-agent courier, floor 15 tok/s): time-under-floor −52%, deep sub-floor −83%, delivery p50 −43%, 100% task score. **Zero-conf GPU sharing.** Instances on one GPU discover each other over shared memory — no ports, no config — and split compute by `--gpu-share-weight`. Measured (3090): weights 2:1 → 71.7/36.2 tok/s; holds at `--parallel 4` and under MTP. Idle peers cost nothing (solo = full speed), crashes age out in 3 s; `GET /gpu/peers` shows live shares + busy %. From c5 every release ships per-platform side-load DSOs: `dso/<ver>/` with Linux x86_64 + sbsa `.so` and a Windows `.dll`. https://huggingface.co/ManniX-ITA/opencoti-llamafile
View all activity

Organizations

None yet