|
Download docs/README.md from staccs/ds4-winhip: direct link, hf CLI and curl.
- Browser
- Download file 12 kB
-
https://huggingface.co/staccs/ds4-winhip/resolve/main/docs/README.md
- Command line
-
hf download hf://staccs/ds4-winhip/docs/README.md
-
curl -L -o README.md https://huggingface.co/staccs/ds4-winhip/resolve/main/docs/README.md
12 kB
Documentation
Start with the repository README. Use the guides below for operation and development; dated reports retain their recorded commit, artifact and workload. They do not describe live processes or qualify a new build.
Start here
| Goal | Guide |
|---|---|
| Build and run | Quick start |
| Select a model and download its artifact | Model Zoo and family contracts |
| Understand the performance examples | Performance references |
Operation
| Guide | Covers |
|---|---|
| Serving contract | Shared options, requested/effective/qualified plans, owner/worker and reuse |
| Generated capabilities | Runtime lanes, modes, bounds and recorded qualification |
| Chat templates | Official input assets, normalization and continuation |
| API matrix | HTTP objects, routing, media forms and unsupported requests |
| Memory guard | Large-model admission, job limits and cleanup |
| SSD expert streaming | Bounded GLM expert cache, options and memory policy |
| GLM Prefill on Spark, 2026-10-08 | Large batches, partial offload, numerical contracts and measured adoption |
Conversion tools
| Guide | Covers |
|---|---|
| VQ calibration | tools/vq_calib/: activation statistics, importance-weighted Lloyd for shared-codebook GVQ, 9/10-bit GVQ v2, the A.X-K2 VQ pipeline |
Models
Choose the exact artifact from the Model Zoo. The family guide owns shared state contracts and the Qwen/GLM/K2 recipes. Other recipes and dated gates retain their hardware and workload limits.
| Guide | Covers |
|---|---|
| Model families | Architecture selectors, native state, context, snapshots and reuse |
| Qwen | Q5, images, owner/cache setup and YaRN |
| Qwen derivatives | Uncensored, Swift and Darwin artifact-specific evidence |
| Qwen FP8 PLE | Sidecar selection, KV compatibility and paired card sweeps |
| Qwen benchmark reference | Consolidated source limits, protocols, campaign numbers, candidate adoption and optimization target |
| GLM | Q2 + vision; recorded serial 2K qualification |
| GLM Uncensored | Mixed recipe, SSD/bank contracts and current verification boundaries |
| GLM VQ | GVQ1 format concept, GB10 execution paths, numerical contract and optimization log |
| K2 | MQ87, IFM tools, one-bank serving and disk limits |
| dots3 | Separate opt-in text banks and serial MTP |
| Bonsai | Pinned ternary PQ2_0, CPU reference and serial CUDA |
| Inkling | MQ85GB/MTP, media input and serial serving |
| Step | MQ83, opt-in banks/MTP, partial fork and disk KV |
| Ling | MQ-Q5, images, persistent banks and YaRN 256K |
| Ling-3.1 VQ | GVQ1 release directory (v2 recipe), FP8 latent KV, embedded MTP, 262K banks beside a 106 GiB trunk, 2K–64K sweep |
| Beam-1.0 VQ | GVQ1 release directory, FP8 full-layer KV, Harmony header forcing, DFlash2 drafter and its losslessness gate, 1M in one bank, 2K–64K sweep |
| MiMo RL | Recorded mixed serving, bank reuse and memory limits |
| MiMo MOPD | Separate artifact, text performance and DFlash boundaries |
| Naive | MQ87, banks, disk KV, long-context gates and drafter limits |
| IQuest-Q1 | Canonical mixed weights, hybrid Q8 KV, embedded MTP and pending Spark gates |
| DeepSeek V4.1 Flash | Serial lane, metadata context, sidecar/engram mounts and the embedded MTP contract |
| DeepSeek V4.1 Flash (ds41) | Artifact, 40-layer shape, serving contract, gates and recorded benchmarks |
| DeepSeek V4.1 Flash (plan) | Proposal: the vq8sh14 artifact, the C engine as source of truth, the gap inventory and the golden-set gate |
| DeepSeek V4.1 post-train (plan) | The â‘¢ producer, judge, apply and supervising HTTP API: units, ports and gates |
Development
| Guide | Covers |
|---|---|
| Contributing | Ordered host checks, native gates and evidence requirements |
| Repository instructions | Shared development and agent rules |
| Architecture | Rust/native ownership, crates and source layout |
| FFI contract | Opaque inference boundary |
| ds4-perf | Inspect, scout, compare, optimize and serving workloads |
| Optimization playbook | Execution-path diagnosis and numerical proof |
| Speed benchmarks | Manual sweep CSVs and plotting |
| Lineage | Provenance, development context and selective upstream ports |
Recorded evidence
Release ledgers
| Ledger | Recorded scope |
|---|---|
| v0.1.3 | Common serving controls and scoped P0–P4 qualification |
| v0.1.2 | Shared Jinja and Inkling checkpoint |
| v0.1.1 | Performance workflow and FP8 PLE |
| v0.1.0 | Independent Rust-host production baseline |
| K2/GLM boundary gates | Native lifecycle, short restart and context checks |
Performance and serving reports
The performance guide explains the README examples. These reports preserve successful gates, rejected experiments and unresolved limits.
- Qwen: September 4 long prefill, September 6, September 7 prefill, September 7 image/text/agent, September 14 draft/prefix, benchmark reference, October 2 Darwin images and follow-up rounds.
- Solar: September 7, September 12 rounds 1–4, September 12 round 5, September 14 capped-clock, August 21 partial reuse.
- Motif: historical throughput and integration gates, August 22 partial reuse.
- GLM VQ: packed restore, register MMA prefill, nonexpert Q4_K candidate, cooperative Q4 decode, direct VQ expert decode, fused FP8 projections, selective KDA Q4 assessment, KDA Q4 speed baseline, direct BF16 KV handoff, latent decode attention, sparse attention through 8K, VQ banks and snapshots, selective DSA Q4 assessment, prefill chunk assessment, workspace allocation assessment.
- dots3: September 6 optimization.
- K2: September 5, continuation campaign.
- Inkling: rounds 1–12, 13–15, 16–18, 19–21, 22–24, 25–27, September 22.
- Step: initial integration, BASE campaign, SWA/MTP rounds, capped-clock campaign.
- Ling: GB10 campaign and long-context sweep, raw CSVs and receipt.
- MiMo RL: September 23, September 24, September 25 serving, September 26 text, September 26 media.
- MiMo MOPD: September 30 controls and proofs.
- Naive: October 1 additional rounds, September 30 campaign, serving, long-context gates.
- Bonsai: serving, prefill, split-K decode, session state, CUDA graph, CPU reference.
- DeepSeek V4.1: vision review: image KV, tools and bounded numerical gates, October 14, vision weight staging: paired TTFT, decode costs and multi-image gates, October 14, warm-path campaign: the asset-open cache, October 13, 2K–64K Prefill/Decode tables and graph, October 13, capped-clock sweep and review qualification, October 13, five-case benchmark, October 13, prefill/decode, October 11, speed sweep, October 12, raw 22-frontier CSV, r6 build CSV, engine/fork compare chart.
Migration and inherited history
- Migration evidence: frozen C oracle, host parity, shared engine gaps and the completed repository split.
- Model-family history: inherited C integration and Motif/Solar workload reports.
Completed migration plans and old process handoffs remain in the pre-cleanup documentation tree. Recorded PIDs are historical.
Design records
- Qwen image input contract: original design; use the family guide and dated image gates for implemented scope.
- W2A16 fused-dequant GEMM proposal: proposed work, not a released path.
- DeepSeek model-card synopsis: upstream model information, separate from DS4 artifact and runtime validation.