Text Generation
GGUF
English
Chinese
Japanese
qwen2
sortmark
slm
small-language-model
edge-computing
global-trade
network-security
data-compliance
zero-trust
cio-insights
cto-insights
information-engineering
security-architects
business-analysts
dpo-governance
policy-as-code
json-schema
structured-outputs
qwen2.5
q4_k_m
llama.cpp
ollama
offline-inference
beta
conversational
Instructions to use sortmark/SORTMARK-DataAudit-0.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use sortmark/SORTMARK-DataAudit-0.5B with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M # Run inference directly in the terminal: llama cli -hf sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M # Run inference directly in the terminal: llama cli -hf sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M
Use Docker
docker model run hf.co/sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use sortmark/SORTMARK-DataAudit-0.5B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "sortmark/SORTMARK-DataAudit-0.5B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "sortmark/SORTMARK-DataAudit-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M
- Ollama
How to use sortmark/SORTMARK-DataAudit-0.5B with Ollama:
ollama run hf.co/sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M
- Unsloth Desktop
- Pi
How to use sortmark/SORTMARK-DataAudit-0.5B with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use sortmark/SORTMARK-DataAudit-0.5B with Docker Model Runner:
docker model run hf.co/sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M
- Lemonade
How to use sortmark/SORTMARK-DataAudit-0.5B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M
Run and chat with the model
lemonade run user.SORTMARK-DataAudit-0.5B-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use sortmark/SORTMARK-DataAudit-0.5B with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use sortmark/SORTMARK-DataAudit-0.5B with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "sortmark/SORTMARK-DataAudit-0.5B:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
File size: 2,543 Bytes
693c502 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 | {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "DataComplianceAuditContract",
"description": "Deterministic JSON Schema for regulatory compliance, PII inspection, and dynamic data masking.",
"type": "object",
"required": [
"audit_id",
"risk_level",
"pii_detected",
"findings",
"compliance_frameworks",
"masking_recommendations",
"verdict"
],
"properties": {
"audit_id": {
"type": "string",
"pattern": "^AUDIT-[A-Z0-9_-]+$"
},
"risk_level": {
"type": "string",
"enum": [
"LOW",
"MEDIUM",
"HIGH",
"CRITICAL"
]
},
"pii_detected": {
"type": "boolean"
},
"findings": {
"type": "array",
"items": {
"type": "object",
"required": [
"field_name",
"category",
"sensitivity",
"sample_pattern"
],
"properties": {
"field_name": {
"type": "string"
},
"category": {
"type": "string",
"enum": [
"NATIONAL_ID",
"FINANCIAL_CARD",
"MOBILE_PHONE",
"PASSPORT",
"LOCATION_GPS",
"DEVICE_IDENTIFIER",
"PERSON_NAME"
]
},
"sensitivity": {
"type": "string",
"enum": [
"PUBLIC",
"INTERNAL",
"CONFIDENTIAL",
"RESTRICTED"
]
},
"sample_pattern": {
"type": "string"
}
}
}
},
"compliance_frameworks": {
"type": "array",
"items": {
"type": "string"
}
},
"masking_recommendations": {
"type": "array",
"items": {
"type": "object",
"required": [
"field_name",
"strategy"
],
"properties": {
"field_name": {
"type": "string"
},
"strategy": {
"type": "string",
"enum": [
"DYNAMIC_PARTIAL_MASK",
"SHA256_HASH",
"TOKENIZE",
"TRUNCATE",
"DROP"
]
}
}
}
},
"verdict": {
"type": "string",
"enum": [
"PASSED",
"WARNING",
"REJECTED_REMEDIATION_REQUIRED"
]
}
}
} |