Instructions to use plantwaterer/kudhab with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use plantwaterer/kudhab with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="plantwaterer/kudhab") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("plantwaterer/kudhab") model = AutoModelForMultimodalLM.from_pretrained("plantwaterer/kudhab", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use plantwaterer/kudhab with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "plantwaterer/kudhab" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "plantwaterer/kudhab", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/plantwaterer/kudhab
- SGLang
How to use plantwaterer/kudhab with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "plantwaterer/kudhab" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "plantwaterer/kudhab", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "plantwaterer/kudhab" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "plantwaterer/kudhab", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use plantwaterer/kudhab with Docker Model Runner:
docker model run hf.co/plantwaterer/kudhab
𒋻𒇥 (kudhab)
Licence: the weights and the code in this repository are released under Apache-2.0.
Decision Index 0.3 public score: 73.96 (self-run; raw 80.03, breadth 73.54).
- A decision model: for every typed question of a request (a choice, or yes / no, among the options given) it returns a probability for each option, read in one forward pass; nothing is generated.
- The weights are a fine-tune of a uniform average of four fine-tunes of one Gemma 4 31B starting point. Every stage was trained to match the output probabilities of MachinePulseAI/karpo-decision-31B on our training questions (distillation from its outputs). This model contains no karpo-decision weights and was not initialised from them.
- It is read with karpo-decision's open inference code: its request layout, attention and readout, and its Decision Index engine
karpo_decision.serve.engine:DecisionIndexEngine, all by MachinePulse (machinepulse-ai/karpo-decision, Apache-2.0). An unmodified copy at commit a6fa38bf4e8de9dbdb9491fb461b9b320a2e2118 is inkarpo-decision/. - Kit: apolinario/decision-index @ 9eb2dbe2a358004c8782c66e40a83ac07b953fec.
Usage
REPO=$(python -c "from huggingface_hub import snapshot_download; print(snapshot_download('plantwaterer/kudhab'))")
pip install "torch>=2.14" "transformers>=5.17" "safetensors>=0.6" # our runs: torch 2.14.1+cu130, transformers 5.21.0, Python 3.12
pip install "decision-index @ git+https://github.com/apolinario/decision-index@9eb2dbe2a358004c8782c66e40a83ac07b953fec"
PYTHONPATH="$REPO/karpo-decision/src" python -m decision_index run --engine karpo_decision.serve.engine:DecisionIndexEngine \
--option checkpoint="$REPO" --option max_length=32768 --compact --out runs/kudhab
pip install "$REPO/karpo-decision" installs the same package instead of using PYTHONPATH. In Python:
from karpo_decision.decider import Decider
decider = Decider.from_pretrained(REPO, device="cuda")
decider.decide({"state": "...", "questions": {"q": {"type": "choice", "instructions": "...", "criteria": {"a": "...", "b": "..."}}}})
The repository is a karpo-decision checkpoint: the backbone's weights and config.json, readout.safetensors, decision_config.json
and the tokenizer files.
Inference
All of this is karpo-decision's design and code, used as published:
- Request layout
karpo-blocks-v1: a system turn, then one shared block (the list of questions, then the context), then one branch per question: the question, its options labelled with one-token codes (A, B, ..., up to 255 options) and the answer slotAnswer: (. The pieces are tokenised separately. - Attention: bf16. In the 10 global-attention layers the shared block attends to itself and each branch to the shared block and to itself, in both directions; branches never see each other. The 50 sliding-window layers are causal within their 1,024-token window.
- Readout: at the last token of each branch, the hidden state times a 255 x 5,376 fp32 matrix (the rows of this model's language-model
head, tied to its input embeddings, for the 255 code tokens, as karpo-decision's
checkpoint.initializebuilds it; not trained), soft-capped (30 tanh(x / 30)); the question's probabilities are the softmax at temperature 1 over its options' codes. - Long requests: a request over
max_lengthtokens is split by questions, each part repeating the whole shared block; a question that does not fit inmax_lengthtokens with its state is refused, never truncated. Requests with images are refused (text only). - Engine options:
view(joint, the default;reversed, which averages each choice question whose options may be reordered over their order and the reverse;single) andmax_length(karpo-decision's default 32768). The public run used viewjointwithmax_length32768. - Hardware: one GPU; the weights are 62.5 GB in bf16. On one RTX PRO 6000 (96 GB) the peak GPU memory was 73.6 GiB at max_length 32,768 (nvidia-smi, on the board's latency sample).
Latency (one RTX PRO 6000 Blackwell, 1,000-request sample)
On the board's 1,000-request public latency sample, one request at a time in one process, with karpo-decision's plain attention path
(without its optional blackwell kernels), view joint and max_length 32,768, per request the median of two runs; model load and
warm-up (13 s) are not included. The 4 requests of the sample that this setting refuses are not in the figures.
| median | mean | p80 | p95 | requests over 1 s | peak GPU memory (nvidia-smi) |
|---|---|---|---|---|---|
| 92 ms | 246 ms | 237 ms | 523 ms | 31 | 73.6 GiB |
How it was made
- Starting point: a uniform average (0.5 / 0.5) of plantwaterer/munku (Apache-2.0) and tattrongvu/Jeffan-31b-1710 at revision cebfa86c2bed2b5c83b12e44de4374b20c87fd79, whose LICENSE file and card state Apache-2.0 (the repository was CC BY-NC 4.0 until 2026-10-22; this model uses the Apache-2.0 revision above). Both are fine-tunes of google/gemma-4-31B-it (Apache-2.0).
- Teacher: MachinePulseAI/karpo-decision-31B by MachinePulse, at
revision d69cdd57eab23271cbe65236ca8a00d80d7bb8c7, released under the MachinePulse Community License. It was run as it ships, in its
own inference code (view
joint, temperature 1), on our training questions in its request format; its output probabilities over each question's options are the training targets. Its licence, section 8(f), reads: "A model trained only on outputs generated by the Software or a Derivative Work, and which does not incorporate any of their model weights, is not a Derivative Work." No karpo-decision weights were used to initialise, merge into or train this model. - Stage 1, four fine-tunes (students), each from the starting point, in karpo-decision's request layout, attention and readout as above
(karpo-decision's modules, unmodified, inside our own trainer). Trained: the linear projections of the 60 decoder layers
(29,286,727,680 parameters); embeddings, norms and readout frozen. AdamW (0.9, 0.95), learning rate 1e-6 with 100 warm-up steps
and a cosine decay to 0.1 of it, 64 questions per step, bf16 compute over fp32 master weights, gradient clipping 1.0.
- Three (seeds 1, 2 and 3) on the 359,747 questions with a teacher target, loss = KL divergence from the teacher's distribution alone (no gold labels), one pass = 5,622 steps; the first two taken at step 2,811, the third at the end of the pass.
- One (seed 1) on all 420,731 questions, one pass = 6,574 steps: 0.8 x the KL term + 0.2 x cross-entropy with the gold answer on questions with a teacher target, cross-entropy with the gold answer on the other 60,984.
- Stage 1 average: the uniform average (0.25 each) of the four students' weights.
- Stage 2 (this model): from that average, one pass of 2,152 steps over 137,708 questions, loss = KL divergence from the teacher's distribution alone, learning rate 5e-7 with 100 warm-up steps and a cosine decay, 64 questions per step, seed 1, the same trained parameters as stage 1. The questions: 68,854 drawn at random from stage 1's distillation questions; 33,998 of the 60,984 training questions that had no teacher target in stage 1, now converted to the teacher's request format and labelled by it; and 34,856 new questions (listed under Training data), each with the teacher's output probabilities as target. The readout is then built from the model's embeddings as above.
Training data
Weights it starts from. munku's card lists munku's training data and those of the models it builds on (plantwaterer/eshbar-lulilla and plantwaterer/liquamen); those carry over. The card of Jeffan-31b-1710 (at the revision above) does not describe its training data.
Questions used for the four students: 420,731. These are munku's 384,663 training questions, less 130 synthetic rows (127 procedural typed-decision rows and 3 synthetic workflow rows), plus 36,198 added for this model (train splits where the source has one; questions that match the Decision Index public suite or our held-out sets removed):
| source | questions added | licence, as its authors state it |
|---|---|---|
| VAST (Allaway & McKeown, 2020), train | 9,000 | states no licence |
| ACOS (Cai et al., 2021), laptop and restaurant train | 7,998 | states no licence |
| When2Call (NVIDIA), train_pref | 6,000 | CC BY 4.0 |
| SuperGPQA (M-A-P Team, 2025), items not in munku's training questions | 6,438 | ODC-BY 1.0 (its card asks users to respect the licences of the datasets it draws on) |
| RAGTruth, train responses of open models | 5,680 | MIT (annotations); passages from MS MARCO (non-commercial research terms), the Yelp Open Dataset and CNN/DM |
| Habermas Machine comparisons (Google DeepMind), training split | 1,082 | CC BY 4.0 |
Some sources carry non-commercial or research-only terms, and some state no licence (counts of the 420,731 questions):
- Non-commercial: ANLI (Nie et al., 2020; CC BY-NC 4.0; 702); AmericasNLI (CC BY-SA 4.0), a translation of XNLI (Conneau et al., 2018; CC BY-NC 4.0; 11); x-stance (Vamvas & Sennrich, 2020; CC BY-NC 4.0; 400); the Amazon Multilingual Counterfactual Dataset (O'Neill et al., 2021; CC BY-NC 4.0; 400); and, within DISRPT, the Potsdam Commentary Corpus (CC BY-NC-SA; 160), the Russian RST Treebank 1.0 (CC BY-NC-SA 4.0; 105) and the Persian RST corpus (CC BY-NC; 3). RAGTruth's annotations are MIT, but its passages come from MS MARCO, whose terms limit use to non-commercial research, the Yelp Open Dataset and CNN/DM (6,493 = 813 + 5,680 added).
- Research use: CSTNews (Cardoso et al., 2011; 160), "available for download and use for research purposes".
- States no licence: ACOS (Restaurant-ACOS and Laptop-ACOS, Cai et al., 2021; 9,615 = 1,617 + 7,998 added), VAST (Allaway & McKeown, 2020; 9,702 = 702 + 9,000 added), MultiRC (Khashabi et al., 2018; 555), NLI4CT (Jullien et al., 2023; 702) and SciDTB (Yang & Li, 2018; 1). No licence in the DISRPT distribution: the RST Basque TreeBank, the Dutch Discourse Treebank and the RST Spanish Treebank (160 each).
Questions added in stage 2: 34,856 (train splits; questions that match the Decision Index public suite or our held-out sets removed), besides the 33,998 earlier training questions newly converted to the teacher's request format:
| source | questions | licence, as its authors state it |
|---|---|---|
| MultiRC (Khashabi et al., 2018), train, as select-all questions (2,095 items) | 11,217 | states no licence |
| GSM8K, train; the other options are the answers of other train problems | 10,000 | MIT |
| BANKING77 (PolyAI), train, 77-way | 8,638 | CC BY 4.0 |
| GoEmotions, train, comments with two or more labels, as select-all questions (825 items) | 5,001 | Apache-2.0 |
With these, the MultiRC questions (states no licence) total 11,772 (555 in stage 1, 11,217 in stage 2).
The other sources of munku's training questions, as munku's card lists them:
- Mathematics: the DeepMind Mathematics Dataset (Saxton et al., 2019), problems generated afresh with its own generator (Apache-2.0); AQuA-RAT (Ling et al., 2017; Apache-2.0) and MATH (Hendrycks et al., 2021; MIT) train splits.
- Medicine and logic: MedQA (Jin et al., 2021; MIT) and FOLIO (Han et al., 2022; CC BY-SA 4.0 in the authors' GitHub repository, MIT on their Hugging Face page) train splits.
- Science questions: SuperGPQA (M-A-P Team, 2025; ODC-BY 1.0), released as a single split. Its answer keys were checked with Qwen3-32B (Apache-2.0).
- Tools: Toucan-1.5M (Xu et al., 2025; Apache-2.0): MCP tool schemas with questions and trajectories written by open-weight models, recast as tool-decision questions.
- Fact checking, clinical evidence, intents and products: EX-FEVER (Ma et al., 2024; MIT), with English Wikipedia evidence (CC BY-SA); Evidence Inference 2.0 (DeYoung et al., 2020; MIT), annotator evidence spans only; MTOP (Li et al., 2021; CC BY-SA 4.0); MultiDoGO finance intents (Peskov et al., 2019; CDLA-Permissive-1.0); WANDS (Chen et al., 2022; MIT), labels mapped to the ESCI classes. Corr2Cause (already listed) contributed further train-split items.
- Teacher targets: soft targets from MiMo-V2.6-Pro-RL (Xiaomi; MIT) on 5,354 questions (PubMedQA artificial-label items and one synthetic family).
- Synthetic questions generated by the author (no third-party data), several in the request formats of the Decision Index kit (MIT); the phishing-email questions use the question wording of jev-phishing-bench (MIT). Task formats follow BIG-Bench Hard, MuSR, CLadder, CRUXEval and SATA-Bench; no items from those benchmarks are used.
- Further NLP datasets (train splits, a few hundred questions each), each under its own licence as stated by its authors; where the copy used on the Hub is tagged differently, the authors' statement is given:
Benchmarks with a train split contributed train-split records only. SuperGPQA has no separate train split; its use is described above.
Limits
- Context: 32,768 tokens per sequence (
max_length32768); in the public run 403 of 140,620 requests had a question that does not fit with its state and were refused; the same 403 requests as in karpo-decision-31B's own published run; nothing was truncated. - Text only.
- Downloads last month
- 29