Instructions to use Devseis/endpoint-auditor-0.5b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Devseis/endpoint-auditor-0.5b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Devseis/endpoint-auditor-0.5b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Devseis/endpoint-auditor-0.5b")
model = AutoModelForCausalLM.from_pretrained("Devseis/endpoint-auditor-0.5b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Devseis/endpoint-auditor-0.5b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Devseis/endpoint-auditor-0.5b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Devseis/endpoint-auditor-0.5b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Devseis/endpoint-auditor-0.5b

SGLang

How to use Devseis/endpoint-auditor-0.5b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Devseis/endpoint-auditor-0.5b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Devseis/endpoint-auditor-0.5b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Devseis/endpoint-auditor-0.5b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Devseis/endpoint-auditor-0.5b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Devseis/endpoint-auditor-0.5b with Docker Model Runner:
```
docker model run hf.co/Devseis/endpoint-auditor-0.5b
```

Devseis Endpoint Auditor 0.5B (v0.1)

A small language model fine-tuned by Devseis to write endpoint audit findings for ISO 27001, GDPR and the EU AI Act from collected evidence. It runs on the device (WebLLM in the Devseis Endpoint Auditor desktop app and phone check), so audit evidence never leaves the computer or phone.

v0.1: Pilot: 988 balanced examples from dataset v0.1 (Windows, Linux, macOS). Proves the pipeline; superseded by v0.3.

What the model does — and does not do

Code collects the evidence and decides every verdict (Compliant / Non-Compliant / …) and every vulnerability match. The model explains each finding, rates the risk, writes fix and verification steps for the device's OS, maps findings to ISO 27001 / GDPR / AI Act references, writes the executive summary, and classifies AI tools under the EU AI Act. Every answer is checked against the evidence (numbers, check id, status, references, tool names) before it is used; if a check fails, the app uses a built-in template instead. Details: docs/MODEL_LLM.md.

Use with transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "Devseis/endpoint-auditor-0.5b"
tok = AutoTokenizer.from_pretrained(repo, revision="v0.1")
model = AutoModelForCausalLM.from_pretrained(repo, revision="v0.1")

The LoRA adapter alone is in lora/ (apply it to Qwen/Qwen2.5-0.5B-Instruct with PEFT). Browser builds: Devseis/endpoint-auditor-0.5b-q0f16-MLC (desktop), Devseis/endpoint-auditor-0.5b-q4f16_1-MLC (phones) and Devseis/endpoint-auditor-0.5b-q4f32_1-MLC.

Prompt format

Retrieval-style prompts built by app/renderer/auditor-core.js (identical to training/build_dataset.py), with the system prompt in that file. Tasks: finding, summary, ai_classification; the model answers with one JSON object.

Training


Base model	Qwen/Qwen2.5-0.5B-Instruct (Apache-2.0)
Data	Devseis/endpoint-auditor-synthetic revision `v0.1` (synthetic, CC BY 4.0)
Examples	988 balanced over task, OS and status (2048 tokens)
Method	LoRA r=16, alpha=32 (8.8 M trainable parameters), lr 2e-4, 1 epoch, 124 steps of 8 examples
Hardware	CPU only: a 4-core Intel MacBook Pro (PyTorch 2.2, eager attention)
Final validation loss	0.0173

Evaluation

Held-out test examples (training/evaluate.py, greedy decoding). Grounded = passes the same faithfulness check the app uses; fields match = check id / status / risk (findings), score and top gaps (summaries), tools and approval flags (AI).

Base model (Qwen2.5-0.5B-Instruct, not fine-tuned):

Task	n	Valid JSON	Grounded	Fields match
ai_classification	5	0%	0%	0%
finding	21	0%	0%	0%
summary	4	0%	0%	0%
all	30	0%	0%	0%

This model (v0.1):

Task	n	Valid JSON	Grounded	Fields match
ai_classification	5	100%	100%	100%
finding	21	100%	100%	100%
summary	4	100%	100%	0%
all	30	100%	100%	87%

Limitations

Trained on synthetic evidence; real-world wording varies. The app's grounding check and template fallback guard the report.
0.5B parameters: it phrases and selects retrieved facts well; it is not reliable for arithmetic or ranking, so the app supplies scores and top gaps from code.
Not legal advice and not a certification. ISO 27001 control titles are cited; the standard's text is not reproduced.

Licence and citation

Apache-2.0, by Devseis. Please credit "Devseis Endpoint Auditor by Devseis".

@software{devseis_endpoint_auditor_model_2026,
  author  = {{Devseis}},
  title   = {Devseis Endpoint Auditor 0.5B},
  year    = {2026},
  version = {v0.1},
  url     = {https://huggingface.co/Devseis/endpoint-auditor-0.5b}
}

Downloads last month: -

Safetensors

Model size

0.5B params

Tensor type

F16

Model tree for Devseis/endpoint-auditor-0.5b

Base model

Qwen/Qwen2.5-0.5B

Finetuned

Qwen/Qwen2.5-0.5B-Instruct

Adapter

(869)

this model

Devseis
/

endpoint-auditor-0.5b