Instructions to use Kukie-AI/Kause-K8s-27B-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Kukie-AI/Kause-K8s-27B-FP8 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Kukie-AI/Kause-K8s-27B-FP8") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("Kukie-AI/Kause-K8s-27B-FP8") model = AutoModelForMultimodalLM.from_pretrained("Kukie-AI/Kause-K8s-27B-FP8", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Kukie-AI/Kause-K8s-27B-FP8 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Kukie-AI/Kause-K8s-27B-FP8" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kukie-AI/Kause-K8s-27B-FP8", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Kukie-AI/Kause-K8s-27B-FP8
- SGLang
How to use Kukie-AI/Kause-K8s-27B-FP8 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Kukie-AI/Kause-K8s-27B-FP8" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kukie-AI/Kause-K8s-27B-FP8", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Kukie-AI/Kause-K8s-27B-FP8" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kukie-AI/Kause-K8s-27B-FP8", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Kukie-AI/Kause-K8s-27B-FP8 with Docker Model Runner:
docker model run hf.co/Kukie-AI/Kause-K8s-27B-FP8
Kause-K8s-27B-FP8
English | 한국어
Kause finds the cause.
Current version: v0.2 (training round 2). Later rounds are pushed to this same repo and tagged (v0.2, v0.3, ...).
A Kubernetes troubleshooting model: Qwen3.8-27B fine-tuned (LoRA, round 2) to find the root cause of a broken workload by calling read-only kubectl-style tools, merged and stored in the same FP8 format as the official Qwen/Qwen3.8-27B-FP8. It fits on one 48 GB GPU (L40S, RTX 6000 Ada) with vLLM.
What it does
Given a question like "Why is the checkout pod restarting?" and a set of read-only tools (list, describe, logs, events), the model investigates the cluster step by step and returns:
- the root-cause object (kind / namespace / name),
- the cause category (for example
bad-command,probe-misconfig,oom,rbac,selector-mismatch), - evidence quoted from tool output,
- suggested fix commands for a human to review.
It was trained with Korean-language system prompts and explanations, so it answers in Korean by default. English questions work, but we have not measured them.
Evaluation
All numbers come from the same harness, the same agent (commit 51b380f), up to 50 requests per run, and FP8 on one L40S with vLLM v0.32.0. Each problem was run 3 times. The metric is root-cause accuracy: a run counts only when both the cause object and the cause category are correct. The base model is the official Qwen/Qwen3.8-27B-FP8.
| Set | What it is | Base | Round 2 | Change (95% interval) |
|---|---|---|---|---|
| A | 44 problems from public benchmarks (30 clusters), never used for training | 52.3% | 58.3% | +6.1 pp (+0.8, +12.0) |
| B | ITBench, 14 problems | 54.8% | 69.0% | +14.3 pp (+2.4, +28.6) |
| C | Decoy variants of a public set (33 problems × 3 rounds), checks for shortcut guessing | 93.3% | 95.6% | +2.4 pp (−1.3, +5.7) |
| D | Held-out seeds of our generator, 20 problems | 35.0% | 51.7% | +16.7 pp (−1.7, +36.7) |
| Hidden set | 12 problems (11 faults + 1 healthy), opened only for confirmation | 78.8% | 75.8% | −3.0 pp (−9.1, 0.0) |
How to read this:
- The verdict was fixed before training. It is based on A with a cluster bootstrap (10,000 resamples, seed 292). The lower bound is above 0, so A counts as improved. Both guard checks also passed: no drop on C, and confidently-wrong answers on A went down (42.4% → 37.1%).
- B: the base value comes from an earlier run of the same setup, and we had looked at these problems before. We report it but do not use it for claims.
- D: same generator family as the training data, so this shows direction only.
- The hidden set is small (11 scored problems, one problem ≈ 9 points). It is used only to check that nothing collapsed (rule: a drop of more than 25 points). It did not collapse, and it shows no gain.
- Healthy-cluster false alarms: 0/3 for both models on the hidden set.
How it was trained
- Collect. The base 27B solved about 1,040 synthetic Kubernetes fault scenarios in throwaway kind clusters through our read-only diagnosis agent. Each scenario was solved 3 times. When all 3 failed, it got one more try with a hint.
- Filter. We kept a run only when the root cause was correct and the model had actually looked up the cause (or a pod it owns) and quoted real tool output. Runs that guessed were dropped. We picked one run per problem, capped hint runs, and dropped near-duplicates. Two reviewers read 30 samples, and 3 records were removed.
- Train. LoRA on
Qwen/Qwen3.8-27B(bf16): rank 32, alpha 16, all linear layers, learning rate 1e-4, batch 16, 2 epochs (70 steps), 32k context, 551 conversations. Training was stopped at step 44 by a pre-set check that turned out to be too small (9 problems). We re-measured with 60 runs, found no drop, and resumed from the step-35 adapter with a fresh optimizer. - Merge to FP8.
W' = W_bf16 + (alpha / r)·B·Afor each adapted layer. The result is re-quantized with the same 128×128 block e4m3 scheme as the official FP8 checkpoint. Untouched tensors are copied byte-for-byte. Seecode/merge_lora_fp8.py.
Training data (551 conversations, 47 fault types)
Each example is one full investigation: the question, the model's tool calls, the real tool outputs, and the final answer. Only the model's own turns (tool calls and the final answer) are trained on; tool outputs are context. That is about 0.66M of 6.5M tokens. Tool calls per conversation: 5.4 on average (1 to 18).
| Group | Count | Examples |
|---|---|---|
| Container start / config errors | 106 | wrong command, missing binary, bad args, image pull failure, bad config value |
| Service connectivity | 74 | selector typo, port mismatch, sidecar port conflict, init waiting for a service |
| Resources / memory | 63 | OOMKilled, emptyDir size eviction, HPA max too low, Job deadline too short |
| NetworkPolicy | 60 | default-deny ingress, wrong namespace selector, egress blocking DNS |
| Missing Secret / ConfigMap values | 46 | missing Secret key, Secret name typo, missing ConfigMap key |
| Probes | 42 | liveness on the wrong port, slow start, readiness on the wrong path |
| Storage | 22 | unsupported RWX volume, missing StorageClass |
| Rollout state | 17 | paused Deployment, replicas 0, hanging init container |
| Admission / permissions | 14 | missing PriorityClass, Pod Security violation, missing ServiceAccount |
| Solved after a hint | 23 | problems the model failed 3 times, then solved with a hint |
| Single bare-Pod faults (round 1) | 84 | simple one-Pod faults collected in round 1 |
All faults were injected by our own scenario generator into throwaway kind clusters, and every answer was produced by the base model itself. No answers from other models or humans are included.
The training data is public: Kukie-AI/Kause-K8s-train-v0.2 (551 conversations, Apache 2.0). The scenario generator is not released. All evaluation problems were excluded from training by a do-not-train check.
Use with vLLM
Serve on one 48 GB GPU, with the same settings as our evaluation:
VLLM_API_KEY=change-me bash examples/serve_vllm.sh
That script runs:
docker run --rm --gpus all --ipc=host -p 8000:8000 \
-v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
vllm/vllm-openai:v0.32.0 \
--model Kukie-AI/Kause-K8s-27B-FP8 \
--served-model-name kause-k8s \
--api-key "$VLLM_API_KEY" \
--gpu-memory-utilization 0.9 --max-model-len 65536 --enable-prefix-caching \
--enable-auto-tool-choice --tool-call-parser qwen3_coder \
--reasoning-parser qwen3 --language-model-only --max-num-seqs 64 \
--default-chat-template-kwargs '{"enable_thinking": false}' \
--override-generation-config '{"temperature": 0.7, "top_p": 0.8, "top_k": 20, "min_p": 0.0, "presence_penalty": 1.5, "repetition_penalty": 1.0}'
Notes:
- Thinking mode is off. That is how the model was trained and evaluated.
--tool-call-parser qwen3_coderis required for tool calls.- Set
--api-keywhenever the port is reachable from outside.
Then ask it about a cluster you can read. examples/diagnose.py is a minimal loop: the model calls read-only tools, and the script runs them with your local kubectl and feeds the output back.
pip install openai
VLLM_API_KEY=change-me python examples/diagnose.py --namespace shop "checkout 파드가 계속 재시작돼요. 원인이 뭐예요?"
Plain chat also works:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="change-me")
r = client.chat.completions.create(
model="kause-k8s",
messages=[{"role": "user", "content": "A Deployment's pods are Pending with 'Insufficient cpu'. What should I check first?"}],
)
print(r.choices[0].message.content)
The model was trained with our own agent's tool set and system prompt. Both are included in every row of the training data. Its tool-calling habits work best with tools shaped like list_resources / describe_resource / get_logs / get_events, as in the example.
Limitations
- Small gain. On unseen problems the gain is about +6 points, with a lower bound near 0. On the hidden set there was no gain.
- Fix commands are not verified. The training data was filtered on diagnosis, not on whether the fix commands work. Some suggested commands fail, for example when they patch immutable fields. Review them before running.
- Evidence habits. It sometimes names the controller after looking only at its pod, and it sometimes shortens quotes with "...". Our strict evidence checker counted these as not grounded.
- Measurement caveats. For A and D, each kind cluster ran the base model first and then round 2 on the same problems. Cleanup passed every time. B and C base values come from earlier runs.
- Kubernetes-only, single-cluster, read-only diagnosis. It is not meant to change clusters on its own.
License and attribution
- Apache 2.0, the same as the base model Qwen/Qwen3.8-27B (Qwen team). The FP8 layout follows Qwen/Qwen3.8-27B-FP8.
- Evaluation problems come from public benchmarks (for example k8s-ai-bench by GKE Labs, Apache 2.0; ITBench; AIOpsLab). They were used only for evaluation, never for training.
한국어
한국어 설명 전체는 README_ko.md에 있습니다.
Credit
This work was supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP), funded by the Ministry of Science and ICT (MSIT), Korea, in 2026 (IITP-2026-AI·SW Maestro).
- Downloads last month
- 81
Model tree for Kukie-AI/Kause-K8s-27B-FP8
Base model
Qwen/Qwen3.8-27B

