synthlora-tier3-kubernetes-internals

This repository contains the best-selected LoRA adapter for kubernetes_internals from the multitier LoRA sweep. Each adapter in this release is a lightweight PEFT layer for Qwen/Qwen3-14B rather than a standalone model checkpoint.

At A Glance

• Base model: Qwen/Qwen3-14B
• Tier: 3 (Domain knowledge)
• LoRA identifier: kubernetes_internals
• Intended behavior: Internal architecture and control-flow of Kubernetes: how the control plane, node components, and core objects actually work. Covers mechanisms, not usage patterns. Excludes application-layer patterns (Helm, Operators beyond mechanism), ecosystem add-ons, and most cloud-specific integrations.
• Teacher model used for synthetic data generation: gemini-3.1-flash-lite-preview
• Training objective: Supervised fine-tuning on synthetic domain-fact and QA demonstrations.
• Best checkpoint metric: mcq_accuracy = 0.6200 at step 192
• Training rows used: 784
• Evaluation rows used: 100

Prominent Caveats

• All probe scores on this card are synthetic and model-judged. Use human spot-checks before relying on the adapter in a real workflow.

Example Generations

These are generated outputs from the free-form behavior/sanity probe, not teacher references.

Behavior Elicitation Example

Prompt:

Which of the following statements accurately describes the operational mechanisms of a Kubernetes cluster regarding resource management and state consistency? Answer directly in 2-4 sentences. Do not list option numbers.

Generated answer excerpt:

A Kubernetes cluster maintains state consistency by utilizing etcd as its primary distributed key-value store, which records all cluster-level configuration data. To manage resource allocation effectively, the cluster relies on the kube-scheduler to assign pods to nodes and the kube-controller-manager to reconcile the actual state with the desired state defined by the user.

Judge score: 4/4

Judge note:

The candidate answer correctly identifies etcd's role in maintaining state consistency and accurately describes the roles of the kube-scheduler and kube-controller-manager in resource management and state

Generic Sanity Example

Prompt:

Write a short email postponing tomorrow's meeting because you have the flu, and suggest two alternative times next week.

Generated answer excerpt:

Subject: Postponement of Tomorrow's Meeting Dear Team, I'm sorry to inform you that I've come down with the flu and will be unable to attend tomorrow's meeting. To ensure we can still move forward effectively, I'd like to propose two alternative times next week: either on Wednesday at 10:00 AM or on Thursday at 2:00 PM. Please let me know which time works best for your schedule, and I'll make sure to be available for the discussion. Thank you for your understanding.

Judge score: 4/4

Judge note:

The email is clear, coherent, directly addresses the prompt, and provides two specific alternative times without any issues.

What This Adapter Should Do

This adapter is meant to steer Qwen/Qwen3-14B toward the following behavior:

Internal architecture and control-flow of Kubernetes: how the control plane, node components, and core objects actually work. Covers mechanisms, not usage patterns. Excludes application-layer patterns (Helm, Operators beyond mechanism), ecosystem add-ons, and most cloud-specific integrations.

• Seed atoms: 100
• Augmentation factor: 8
• Compositional eval depth: 2

How To Prompt It

Ask direct domain questions in the adapter's topic area; this tier is about narrow knowledge steering rather than style transfer.

Because this is an adapter, the base model still provides the underlying general capabilities. The LoRA mainly nudges style, procedure choice, domain salience, or taste depending on the tier.

Evaluation

• Main metric: mcq_accuracy
• What it measures: Accuracy on held-out compositional multiple-choice questions built from domain facts that were not paired together in training.
• Project target bar: mcq_accuracy >= 0.40 on the 100-question held-out probe set.
• Base model on the same held-out probe set: mcq_accuracy = 0.5100
• Gain over base at the selected checkpoint: +0.1100
• Observed best checkpoint for this repo: mcq_accuracy = 0.6200 at step 192

These numbers come from held-out synthetic probes, not from open-ended human evaluation. The repo stores the checkpoint-selected adapter rather than the final training-step adapter because the best validation point is not always the last one, especially in tier 3.

Representative Training Examples

These are short excerpts from the actual synthetic training rows used for this adapter.

Example 1

Training text excerpt:

The Kubernetes API server functions as the sole component authorized to perform write operations against the etcd data store. Every other part of the control plane, including the scheduler and controller manager, as well as all worker node components like the kubelet, must route their interactions with the cluster state through this central API server. This architecture ensures that all state changes undergo consistent validation, admission control, and auditing processes before being persisted.

Metadata:

• Form: declarative_1
• Subtopics: control plane components: API server, etcd, controller manager, scheduler

Example 2

Training text excerpt:

The operator pattern functions by utilizing a custom controller specifically designed to oversee and manage the entire lifecycle of a defined custom resource within a Kubernetes environment. This architectural approach ensures that the resource remains in its intended state through automated oversight and management processes that are triggered by the controller's logic.

Metadata:

• Form: declarative_1
• Subtopics: controllers and reconciliation loops

Representative Held-Out Evaluation Examples

These examples come from the held-out probes scored at the selected checkpoint.

Correct held-out probe

Question:

Which statement accurately describes the operational responsibilities of different Kubernetes components regarding service networking and container lifecycle management?

Options:

1. The iptables mode of kube-proxy manages service load balancing via kernel-level rules, while the kubelet ensures container availability by...
2. The iptables mode of kube-proxy is responsible for monitoring container health and automatically restarting pods that fail their liveness probes.
3. The kubelet implements service load balancing by configuring kernel-level iptables rules to route traffic to the appropriate backend pods.
4. The informer pattern manages service load balancing by maintaining a local cache of all active service endpoints and dispatching traffic events to...

Best-checkpoint prediction: option 1 Gold answer: option 1 Result: correct

Missed held-out probe

Question:

Which of the following statements accurately describes the operational mechanisms of a Kubernetes cluster regarding resource management and state consistency?

Options:

1. The controller's workqueue is responsible for managing pod preemption across nodes to ensure that reconciliation events are processed reliably.
2. etcd relies on a primary-replica replication model rather than the Raft consensus algorithm to ensure data consistency.
3. The scheduler's preemption logic can evict pods across different nodes if necessary, while etcd utilizes the Raft consensus algorithm to maintain...
4. The scheduler's preemption logic is restricted to evicting pods only within the same node to maintain cluster stability.

Best-checkpoint prediction: option 2 Gold answer: option 3 Result: incorrect

Behavior And Sanity Probe Findings

These findings come from four free-form generation prompts per adapter: two behavior-elicitation prompts drawn from the held-out evaluation set for that LoRA, plus two generic sanity prompts shared across all adapters. The base Qwen/Qwen3-14B graded each answer on a 0..4 rubric.

• Probe prompts answered: 4
• Behavior elicitation mean: 3.00 / 4.00
• Behavior elicitation pass rate at score >= 3: 50.0%
• Generic sanity mean: 4.00 / 4.00
• Generic sanity pass rate at score >= 3: 100.0%
• Overall mean across all four prompts: 3.50 / 4.00

Interview Probe Findings

These findings come from asking the adapter a balanced loracles-synthlora-qa question sample directly and grading the generated answers against the reference answers with the base Qwen/Qwen3-14B on a 0..4 rubric. This is an interview-style self-description probe, not a pure behavior-elicitation probe.

• Probe questions answered: 10
• Questions per synergy level: 2
• Mean judge score: 3.00 / 4.00
• Pass rate at score >= 3: 80.0%
• Mean judge score by synergy level: L1 3.00, L2 2.00, L3 3.00, L4 3.50, L5 3.50

Example judged answer:

• Question: Which component in the Kubernetes architecture is permitted to write directly to the etcd storage backend?
• Judge score: 4/4
• Generated answer excerpt: The API server is the only component authorized to write directly to the etcd storage backend. It acts as the central gateway for all cluster state changes, ensuring that all operations are validated and serialized correctly before being persisted.
• Judge note: The candidate answer correctly identifies the API server as the only component authorized to write directly to etcd and explains its role as

Files In This Repo

• adapter_model.safetensors and adapter_config.json: the selected PEFT adapter weights
• tokenizer.json, tokenizer_config.json, chat_template.jinja: tokenizer assets mirrored for easier loading
• best_checkpoint.json: the selected evaluation checkpoint metadata
• eval_log.jsonl: evaluation history across checkpoints
• run_config.jsonl: the training configuration used for this run

Load Example

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3-14B"
adapter_id = "mats-10-sprint-cs-jb/synthlora-tier3-kubernetes-internals"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, adapter_id)

Related Artifacts

The companion synergy Q&A dataset is published at:

• https://huggingface.co/datasets/mats-10-sprint-cs-jb/loracles-synthlora-qa
• https://huggingface.co/collections/mats-10-sprint-cs-jb/loracles-synthlora

Limits

• The adapter is narrow by design; it is not meant to replace the base model.
• Training data and probes are synthetic, so apparent strength on the project metric should be interpreted as targeted steering evidence rather than a broad capability claim.
• Tier 3 is the narrowest and noisiest tier. Gains over base can be modest even when the adapter is genuinely useful.