ManniX-ITA/osync-code
Viewer • Updated • 1 • 17
ManniX PRO
ManniX-ITA
AI & ML interests
None yet
Recent Activity
commentedon a paper about 12 hours ago
Group Entropy-Controlled Policy Optimization repliedto their post 1 day ago
# opencoti-llamafile 0.10.3-c5 — settled admission for multi-agent serving
New cut of the opencoti single-file inference engine (llamafile 0.10.3 / llama.cpp + 87 additive patches). Zero-dependency APE: one executable for Linux, Windows, macOS & BSD.
What's new vs c4:
**PolyKV fan-out — pool from a live session.** `POST /polykv/pools` gains `from_session`/`from_slot`: the shared prefix is snapshotted server-side from the session's cached KV — no tokens resent, token-exact. Ephemeral pools auto-release when orchestrators die mid-round.
**PolyKV P7 — settled admission.** Spawning agents faster than the tps signal settles was oversubscribing pools. Now: a per-pool settle window paces admits just enough for a reliable reading; warming sessions no longer bias the mean; the post-admit forecast uses the measured per-admit drop; idle gaps (agents mid-tool-call) no longer read as free capacity; `guarantee_min_sessions` means a new/nested pool always gets its first agent — capacity checks can never deadlock an orchestrator; the enforced gate applies to new sessions only, with per-request `overcommit`. Benchmark (multi-agent courier, floor 15 tok/s): time-under-floor −52%, deep sub-floor −83%, delivery p50 −43%, 100% task score.
**Zero-conf GPU sharing.** Instances on one GPU discover each other over shared memory — no ports, no config — and split compute by `--gpu-share-weight`. Measured (3090): weights 2:1 → 71.7/36.2 tok/s; holds at `--parallel 4` and under MTP. Idle peers cost nothing (solo = full speed), crashes age out in 3 s; `GET /gpu/peers` shows live shares + busy %.
From c5 every release ships per-platform side-load DSOs: `dso/<ver>/` with Linux x86_64 + sbsa `.so` and a Windows `.dll`.
https://huggingface.co/ManniX-ITA/opencoti-llamafile repliedto their post 1 day ago
# opencoti-llamafile 0.10.3-c5 — settled admission for multi-agent serving
New cut of the opencoti single-file inference engine (llamafile 0.10.3 / llama.cpp + 87 additive patches). Zero-dependency APE: one executable for Linux, Windows, macOS & BSD.
What's new vs c4:
**PolyKV fan-out — pool from a live session.** `POST /polykv/pools` gains `from_session`/`from_slot`: the shared prefix is snapshotted server-side from the session's cached KV — no tokens resent, token-exact. Ephemeral pools auto-release when orchestrators die mid-round.
**PolyKV P7 — settled admission.** Spawning agents faster than the tps signal settles was oversubscribing pools. Now: a per-pool settle window paces admits just enough for a reliable reading; warming sessions no longer bias the mean; the post-admit forecast uses the measured per-admit drop; idle gaps (agents mid-tool-call) no longer read as free capacity; `guarantee_min_sessions` means a new/nested pool always gets its first agent — capacity checks can never deadlock an orchestrator; the enforced gate applies to new sessions only, with per-request `overcommit`. Benchmark (multi-agent courier, floor 15 tok/s): time-under-floor −52%, deep sub-floor −83%, delivery p50 −43%, 100% task score.
**Zero-conf GPU sharing.** Instances on one GPU discover each other over shared memory — no ports, no config — and split compute by `--gpu-share-weight`. Measured (3090): weights 2:1 → 71.7/36.2 tok/s; holds at `--parallel 4` and under MTP. Idle peers cost nothing (solo = full speed), crashes age out in 3 s; `GET /gpu/peers` shows live shares + busy %.
From c5 every release ships per-platform side-load DSOs: `dso/<ver>/` with Linux x86_64 + sbsa `.so` and a Windows `.dll`.
https://huggingface.co/ManniX-ITA/opencoti-llamafileOrganizations
None yet