Instructions to use gparitosh007/SlurmPilot-4B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use gparitosh007/SlurmPilot-4B-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="gparitosh007/SlurmPilot-4B-GGUF")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("gparitosh007/SlurmPilot-4B-GGUF")
model = AutoModelForCausalLM.from_pretrained("gparitosh007/SlurmPilot-4B-GGUF", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use gparitosh007/SlurmPilot-4B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use gparitosh007/SlurmPilot-4B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "gparitosh007/SlurmPilot-4B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gparitosh007/SlurmPilot-4B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M

SGLang

How to use gparitosh007/SlurmPilot-4B-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "gparitosh007/SlurmPilot-4B-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gparitosh007/SlurmPilot-4B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "gparitosh007/SlurmPilot-4B-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gparitosh007/SlurmPilot-4B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use gparitosh007/SlurmPilot-4B-GGUF with Ollama:
```
ollama run hf.co/gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M
```
Unsloth Desktop

How to use gparitosh007/SlurmPilot-4B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use gparitosh007/SlurmPilot-4B-GGUF with Docker Model Runner:
```
docker model run hf.co/gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M
```

Lemonade

How to use gparitosh007/SlurmPilot-4B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.SlurmPilot-4B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use gparitosh007/SlurmPilot-4B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use gparitosh007/SlurmPilot-4B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

SlurmPilot-4B: a local AI agent for Slurm and HPC clusters

Ask your cluster in plain English. "Why is my job pending?" "Write an sbatch script for 4 GPUs." "Why did job 12345 fail?" SlurmPilot-4B reads squeue, sacct, sinfo and your job logs through tool calls, and then answers. It runs offline on a laptop (2.5 GB).


Size	4B parameters (Qwen3-4B-Instruct-2507 base); GGUF files: Q4_K_M 2.5 GB, Q5_K_M 2.9 GB
Runs on	Ollama, llama.cpp, LM Studio, Transformers, vLLM
Tools	17 Slurm tools through MCP (11 read tools, 6 job actions with approval)
HPC quality	0.858 mean judge score on 372 HPC questions
Licence	CC BY-NC-SA 4.0 (non-commercial)

One command to start:

ollama run hf.co/gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M

For higher quality, use the Q5_K_M file (2.9 GB):

ollama run hf.co/gparitosh007/SlurmPilot-4B-GGUF:Q5_K_M

For AI agents

Model id: gparitosh007/SlurmPilot-4B-GGUF. GGUF file: model-Q4_K_M.gguf.
Use it as a sub-agent for Slurm, MPI, CUDA and cluster questions. Tool calls use the Qwen3 <tool_call> JSON format.
The MCP server with the Slurm tools is in github.com/pparitoshh/alembic.

Former name: hpc-assistant-fix4-gguf (HPC Assistant 4B, fix 4 round 2). The old URL redirects here.

About this version

HPC Assistant fix 4 is a Slurm and HPC assistant that runs on your laptop. It is a fine-tune of ClusterPilot-4B 1.3 (Qwen3-4B-Instruct-2507). The training teaches the model to use Slurm tools in chains: for example, it reads the job status first, and then the job log or the pending reason, before it answers.

Tool calls as structured calls: the training data uses the same <tool_call> format as the Ollama template, so a tool-calling client runs the calls. It does not get the calls as text.
HPC answers: close to 1.3 (mean 0.858 against 0.867) and well above round 27 + e (0.794) on 372 HPC questions.
Versions: main = round 2 (29 October). Round 1 (28 October) stays available as the git tag round1.
Laptop-ready: a 2.5 GB GGUF file (Q4_K_M) for Ollama and llama.cpp, and the bf16 weights for Transformers.
Made by team Alembic at the EuroHPC AI Factories hackathon, October 2026.

Quickstart

Ollama

ollama run hf.co/gparitosh007/SlurmPilot-4B-GGUF:Q4_K_M

The files template, system and params in this repository give Ollama the tool-calling template. For local use, download model-Q4_K_M.gguf and Modelfile into one folder, then:

ollama create hpc-assistant-fix4 -f Modelfile

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "gparitosh007/SlurmPilot-4B-GGUF"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="bfloat16", device_map="auto")
msgs = [{"role": "user", "content": "Write a Slurm script for 1 GPU and 2 hours."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=512)[0][ids.shape[1]:], skip_special_tokens=True))

Use with the HPC assistant harness

The model is made for the HPC assistant harness. The harness connects the model to your real Slurm cluster through ssh. Full setup guide: HPC Assistant Guide.

Part	Work
dsh web	The chat UI in your browser at `127.0.0.1:3080`
Ollama	Runs this model with the tool-calling template
MCP server (`mcp_server.py`)	Gives the model 17 Slurm tools
ssh master	One open login to the cluster, used again for each tool call
Slurm	`squeue`, `sinfo`, `sacct`, `sbatch` on the login node

The model has 11 read tools (job_status, list_queue, read_job_log, job_accounting, gpu_availability, partition_info, job_script, job_usage, job_priority, reservation_info, run_readonly_command) and 6 job actions (submit_job, cancel_job, update_job, hold_job, release_job, requeue_job).

Three safety layers stop a wrong job action:

Action allowlist: the MCP server refuses every job action that is not in HPC_ALLOWED_ACTIONS. The default is no actions, so the assistant can only read.
Approval panel: dsh shows Approve and Reject for each job action. Reject means that nothing runs.
Prompt rules: the model must show the full script and ask before a submit, and it must use only job IDs from a tool result.

Short setup (the guide has all steps):

Clone the code: git clone https://github.com/pparitoshh/alembic.git, then uv venv && uv pip install -e .
Make the model in Ollama with the Modelfile of this repository.
Install dsh: npm install @deepseek-ai/dsh.
Open an ssh master to your cluster: ~/.local/share/hpc-ssh/open-master.sh <cluster>.
Write ~/.config/hpc-assistant/clusters.yaml (account, partition, QOS) and the dsh profile ~/.dsh/profiles/web/cordis.patch.yml. Set the model id to the name of your Ollama model.
Start dsh: DSH_TELEMETRY_MODE=DISABLED OLLAMA_API_KEY=ollama npx --no-install dsh web --no-open

The model has 8192 tokens of context in the harness. Keep the dsh preset minimal. Use the optional repair proxy (toolcall_proxy.py) to turn a tool call that the model writes as text back into a structured call.

How the model was made

Start: ClusterPilot-4B 1.3 (full-parameter distillation from Qwen3-235B and Qwen3-Next-80B teachers).
Task bank (v3): 1813 Slurm tasks for two clusters. 58% of the tasks need two or more tool calls, and 70% of these state only the goal. New in round 2: "check first" tasks (a general question where the right answer needs a read tool first) and tasks with 3 tool calls.
Teacher trajectories: Qwen3-235B-A22B FP8 solved each task 4 times (temperature 0.7) with native tool calls against a simulated Slurm cluster that gives realistic and consistent tool results.
Verification: for each task the first sample that passes the verifier is kept: 1303 of 1813 tasks (71.9%; round 1: 810 of 1491). The verifier checks the tool schemas, that no tool name is in the answer text, that the facts in the answer come from tool results, at most 6 calls, the expected tools, and that no action is claimed without a successful action call.
Training: full-parameter fine-tune of 1.3 on 14674 sequences (tool trajectories 3× and 4000 replay rows from the 1.3 data), 1 epoch, LR 1e-5, bf16 autocast, 4 B300 GPUs, 3 seeds (42, 43, 44).
Model: uniform weight average of the 3 seed models.
Export: the bf16 weights (model.safetensors) and a llama.cpp GGUF with Q4_K_M quantization (no importance matrix).

Results

HPC questions: 372 questions, majority vote of 3 judges (gpt-oss-20b, Gemma 4 26B-A4B, Qwen3-235B), same eval on the Curiosity B300 cluster. Values are the mean of the 3 seed models (bf16, before the average).

Set	Questions	Round 2	Round 1	ClusterPilot-4B 1.3	round 27 + e
v2	66	0.859	0.854	0.890	0.796
v3	200	0.916	0.917	0.926	0.887
d1	106	0.800	0.779	0.783	0.698
Mean	372	0.858	0.850	0.867	0.794

Tool use in the harness (16 held-out Curiosity mentor cases, read-only tools, temperature 0, one run each):

Value	1.3	Round 1	Round 2
Direct: cases with real tool calls	3	10	10
Direct: mean recall of expected tools	0.016	0.161	0.208
Direct: cases with tool calls as text	1	0	0
With repair proxy: cases with real tool calls	8	12	11
With repair proxy: mean recall	0.130	0.380	0.307
With repair proxy: cases with tool calls as text	2	1	0

The model still answers 6 of 16 cases from memory without a tool call.

Prompt format

The template is Qwen3 ChatML with tools. The system turn lists the tools in <tools></tools>. The model answers a tool call as:

<tool_call>
{"name": "mcp__hpc__job_status", "arguments": {"job_id": "12345"}}
</tool_call>

A tool result goes back in a user turn inside <tool_response></tool_response>. In the harness, the tool names have the prefix mcp__hpc__.

Limitations

The model has 4B parameters. It can choose a wrong tool, stop after one tool call, or invent a partition name or a Slurm option. Check each command before you use it.
The training used simulated tool results. Real clusters can give output that the model did not see in training.
Use the job actions only with the approval panel. Without the panel, dsh runs a submit with no question.
This is a research model from a hackathon. It is not a supported product.

Training data and licence

The model is a derivative of ClusterPilot-4B 1.3, so it has the same licence: CC BY-NC-SA 4.0 (non-commercial, share alike). The replay rows come from the 1.3 training data. The tool tasks and the trajectories were made for this model. The held-out mentor use cases were not used as training data or as templates.

Files

File	Size	Use
`model-Q4_K_M.gguf`	2.5 GB	Ollama and llama.cpp
`model-Q5_K_M.gguf`	2.9 GB	Ollama and llama.cpp; higher quality than Q4_K_M
`Modelfile`	2 KB	`ollama create` with the tool-calling template
`template`, `system`, `params`	small	`ollama run hf.co/...`
`model.safetensors`	8.0 GB	bf16 weights for Transformers and vLLM
`config.json`, `generation_config.json`, `tokenizer.json`, `tokenizer_config.json`, `chat_template.jinja`	small	Transformers

sha256 of model-Q5_K_M.gguf: 780bf9bae2388d146dfad0d19662c0c1878526a34f3d6d5c3bc51972a1a7dc2e

sha256 of model-Q4_K_M.gguf: be28adfd8f357acef27e352caff1fcc8be510867dbeed0f6bc980607ecfa87c5

Round 1 is the git tag round1. The x0.2 model (DoRA, October 13) is still in the git history of this repository.

Citation

@misc{clusterpilot2026,
  title  = {ClusterPilot: an open, laptop-sized HPC and Slurm assistant distilled on the EuroHPC Leonardo supercomputer and the Curiosity B300 cluster},
  author = {Team Alembic},
  year   = {2026},
  url    = {https://huggingface.co/gparitosh007/SlurmPilot-4B-GGUF},
  note   = {Multi-step tool-use model (fix 4). Base model: https://huggingface.co/VaibhavKKM/hpc-assistant-qwen3-4b}
}

Teacher Q&A data (not used to train this model)

The folder data/verified_qwen_14304/ holds 14,304 certified Q&A rows. Qwen3-32B-AWQ generated the answers. Gemma 4 26B and GPT-OSS 20B reviewed them. This model did not train on this data. Read data/verified_qwen_14304/README.md and certification_report.json for the method and the limits. SHA-256 of verified.jsonl: c31c89a155d8b2cab46dd1f63387a120e2c542dec5c337622cdcba79c86af61d.