Instructions to use PRiyanshu0-1/DPDP-SSense with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PRiyanshu0-1/DPDP-SSense with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="PRiyanshu0-1/DPDP-SSense")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("PRiyanshu0-1/DPDP-SSense", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use PRiyanshu0-1/DPDP-SSense with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf PRiyanshu0-1/DPDP-SSense:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf PRiyanshu0-1/DPDP-SSense:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf PRiyanshu0-1/DPDP-SSense:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf PRiyanshu0-1/DPDP-SSense:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf PRiyanshu0-1/DPDP-SSense:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf PRiyanshu0-1/DPDP-SSense:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf PRiyanshu0-1/DPDP-SSense:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf PRiyanshu0-1/DPDP-SSense:Q4_K_M

Use Docker

docker model run hf.co/PRiyanshu0-1/DPDP-SSense:Q4_K_M

LM Studio
Jan

vLLM

How to use PRiyanshu0-1/DPDP-SSense with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "PRiyanshu0-1/DPDP-SSense"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PRiyanshu0-1/DPDP-SSense",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/PRiyanshu0-1/DPDP-SSense:Q4_K_M

SGLang

How to use PRiyanshu0-1/DPDP-SSense with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "PRiyanshu0-1/DPDP-SSense" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PRiyanshu0-1/DPDP-SSense",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "PRiyanshu0-1/DPDP-SSense" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PRiyanshu0-1/DPDP-SSense",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use PRiyanshu0-1/DPDP-SSense with Ollama:
```
ollama run hf.co/PRiyanshu0-1/DPDP-SSense:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use PRiyanshu0-1/DPDP-SSense with Docker Model Runner:
```
docker model run hf.co/PRiyanshu0-1/DPDP-SSense:Q4_K_M
```

Lemonade

How to use PRiyanshu0-1/DPDP-SSense with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull PRiyanshu0-1/DPDP-SSense:Q4_K_M

Run and chat with the model

lemonade run user.DPDP-SSense-Q4_K_M

List all available models

lemonade list

Atomic Chat

DPDP-SSense / scripts /03_evaluate_models.sh

PRiyanshu0-1

feat(ml): release certified DPDP-SSense SLMs, GGUF, and RAG index

e979960 verified 2 months ago

Raw History Blame Contribute Delete

7.12 kB

#!/usr/bin/env bash
# ==============================================================================
# 03_evaluate_models.sh – Stage 3: Master Certification Orchestrator (DGX Spark)
# ==============================================================================
# SOTA Upgrades Implemented:
# 1. Single CI/CD Entrypoint: Absorbs legacy verify.sh, evaluate.sh, and verify_edge.sh.
# 2. Native Edge Routing: Pass `--edge` to automatically target GGUF binaries via llama.cpp.
# 3. Explicit "No-Fail-Fast" Zone: `set +e` ensures the orchestrator finishes and 
#    generates the Master Scorecard even if individual models catastrophically fail.
# 4. Hardware Env Overrides: Forces TOKENIZERS_PARALLELISM=false for safe VRAM airlocking.
# ==============================================================================

set -eo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ML_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
REPO_ROOT="$(cd "${ML_DIR}/.." && pwd)"

# ─── DGX & PYTORCH ENVIRONMENT HARDENING ──────────────────────────────────────
export PYTHONUNBUFFERED=1
export TOKENIZERS_PARALLELISM=false
export CUDA_DEVICE_ORDER="PCI_BUS_ID"

# ─── DEFAULT CONFIGURATION ────────────────────────────────────────────────────
BACKEND="vllm"
AUDIT_MODEL="../models/audit-model-final"
CHATBOT_MODEL="../models/chatbot-model-final"
AUDIT_LORA="audit"
CHATBOT_LORA="chatbot"
VLLM_URL="http://localhost:8000/v1/completions"
EDGE_MODE=false
EXTRA_ARGS=()

# ─── ARGUMENT PARSING ─────────────────────────────────────────────────────────
while [[ $# -gt 0 ]]; do
    case $1 in
        --backend) BACKEND="$2"; shift 2 ;;
        --audit-model) AUDIT_MODEL="$2"; shift 2 ;;
        --chatbot-model) CHATBOT_MODEL="$2"; shift 2 ;;
        --audit-lora) AUDIT_LORA="$2"; shift 2 ;;
        --chatbot-lora) CHATBOT_LORA="$2"; shift 2 ;;
        --vllm-url) VLLM_URL="$2"; shift 2 ;;
        --skip-run) EXTRA_ARGS+=("--skip-run"); shift ;;
        --edge) EDGE_MODE=true; shift ;;
        -h|--help)
            echo "Usage: bash ml/scripts/03_evaluate_models.sh [options]"
            echo "Options:"
            echo "  --backend <vllm|unsloth|llamacpp>    Backend engine (default: vllm)"
            echo "  --edge                               Auto-configures llama.cpp to test quantized GGUF Edge models"
            echo "  --audit-model <path>                 Path to Auditor model"
            echo "  --chatbot-model <path>               Path to Chatbot model"
            echo "  --skip-run                           Skip inference, aggregate existing JSON reports only"
            exit 0
            ;;
        *) 
            echo "Unknown argument: $1"
            exit 1 
            ;;
    esac
done

# ─── EDGE/GGUF AUTO-CONFIGURATION ─────────────────────────────────────────────
if [[ "$EDGE_MODE" == true ]]; then
    BACKEND="llamacpp"
    AUDIT_MODEL="../models/audit-model-final-gguf"
    CHATBOT_MODEL="../models/chatbot-model-final-gguf"
    EXTRA_ARGS+=("--base-model-label" "GGUF-Q4_K_M-Edge")
fi

# ─── DGX LOGGING SETUP ────────────────────────────────────────────────────────
TIMESTAMP=$(date +"%Y%m%d_%H%M%S")
LOG_DIR="${REPO_ROOT}/logs"
mkdir -p "${LOG_DIR}"
MASTER_LOG="${LOG_DIR}/03_evaluate_models_${TIMESTAMP}.log"

# Setup dual-logging to both terminal and log file safely
exec > >(tee -i "${MASTER_LOG}")
exec 2>&1

echo "════════════════════════════════════════════════════════════════════════════════"
if [[ "$EDGE_MODE" == true ]]; then
    echo "🔧 INITIATING STAGE 3: GGUF EDGE-DEPLOYMENT CERTIFICATION HARNESS"
else
    echo "🏆 INITIATING STAGE 3: FUNCTIONAL & ADVERSARIAL MODEL CERTIFICATION"
fi
echo "════════════════════════════════════════════════════════════════════════════════"
echo "   • Inference Backend: ${BACKEND}"
echo "   • Auditor Model:     ${AUDIT_MODEL}"
echo "   • Chatbot Model:     ${CHATBOT_MODEL}"
echo "   • Master Log:        ${MASTER_LOG}"
echo "════════════════════════════════════════════════════════════════════════════════"

# ─── EXECUTE PYTHON ORCHESTRATOR ──────────────────────────────────────────────
cd "${ML_DIR}/evals"

if [ ! -f "verify.py" ]; then
    echo "❌ ERROR: Master orchestrator 'verify.py' not found in $(pwd)"
    exit 1
fi

# ==============================================================================
# SOTA FIX: THE NO-FAIL-FAST ZONE
# By explicitly suspending 'set -e' (+e), we guarantee Bash will not terminate 
# the pipeline if the SLMs mathematically fail the security bounds. This ensures 
# the Master Markdown Scorecard is ALWAYS compiled and presented to the developer.
# ==============================================================================
set +e

python3 verify.py \
    --backend "${BACKEND}" \
    --audit-model-path "${AUDIT_MODEL}" \
    --chatbot-model-path "${CHATBOT_MODEL}" \
    --audit-lora-name "${AUDIT_LORA}" \
    --chatbot-lora-name "${CHATBOT_LORA}" \
    --vllm-url "${VLLM_URL}" \
    "${EXTRA_ARGS[@]}"

EXIT_CODE=$?

# Re-enable fail-fast safety netting for the remainder of the wrapper
set -e

echo -e "\n════════════════════════════════════════════════════════════════════════════════"
if [ ${EXIT_CODE} -eq 0 ]; then
    echo "✅ STAGE 3 COMPLETE: Models certified across all 18 functional thresholds!"
else
    echo "❌ STAGE 3 COMPLETE: Certification checks encountered threshold infractions."
    echo "   (Review the Master Scorecard. Threshold failures indicate model capability limits,"
    echo "    not pipeline bugs. Adjust SimPO training data to improve.)"
fi
echo "════════════════════════════════════════════════════════════════════════════════"
echo "📁 Execution logs saved to: ${MASTER_LOG}"

exit ${EXIT_CODE}

	#!/usr/bin/env bash
	# ==============================================================================
	# 03_evaluate_models.sh – Stage 3: Master Certification Orchestrator (DGX Spark)
	# ==============================================================================
	# SOTA Upgrades Implemented:
	# 1. Single CI/CD Entrypoint: Absorbs legacy verify.sh, evaluate.sh, and verify_edge.sh.
	# 2. Native Edge Routing: Pass `--edge` to automatically target GGUF binaries via llama.cpp.
	# 3. Explicit "No-Fail-Fast" Zone: `set +e` ensures the orchestrator finishes and
	# generates the Master Scorecard even if individual models catastrophically fail.
	# 4. Hardware Env Overrides: Forces TOKENIZERS_PARALLELISM=false for safe VRAM airlocking.
	# ==============================================================================

	set -eo pipefail

	SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
	ML_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
	REPO_ROOT="$(cd "${ML_DIR}/.." && pwd)"

	# ─── DGX & PYTORCH ENVIRONMENT HARDENING ──────────────────────────────────────
	export PYTHONUNBUFFERED=1
	export TOKENIZERS_PARALLELISM=false
	export CUDA_DEVICE_ORDER="PCI_BUS_ID"

	# ─── DEFAULT CONFIGURATION ────────────────────────────────────────────────────
	BACKEND="vllm"
	AUDIT_MODEL="../models/audit-model-final"
	CHATBOT_MODEL="../models/chatbot-model-final"
	AUDIT_LORA="audit"
	CHATBOT_LORA="chatbot"
	VLLM_URL="http://localhost:8000/v1/completions"
	EDGE_MODE=false
	EXTRA_ARGS=()

	# ─── ARGUMENT PARSING ─────────────────────────────────────────────────────────
	while [[ $# -gt 0 ]]; do
	case $1 in
	--backend) BACKEND="$2"; shift 2 ;;
	--audit-model) AUDIT_MODEL="$2"; shift 2 ;;
	--chatbot-model) CHATBOT_MODEL="$2"; shift 2 ;;
	--audit-lora) AUDIT_LORA="$2"; shift 2 ;;
	--chatbot-lora) CHATBOT_LORA="$2"; shift 2 ;;
	--vllm-url) VLLM_URL="$2"; shift 2 ;;
	--skip-run) EXTRA_ARGS+=("--skip-run"); shift ;;
	--edge) EDGE_MODE=true; shift ;;
	-h\|--help)
	echo "Usage: bash ml/scripts/03_evaluate_models.sh [options]"
	echo "Options:"
	echo " --backend <vllm\|unsloth\|llamacpp> Backend engine (default: vllm)"
	echo " --edge Auto-configures llama.cpp to test quantized GGUF Edge models"
	echo " --audit-model <path> Path to Auditor model"
	echo " --chatbot-model <path> Path to Chatbot model"
	echo " --skip-run Skip inference, aggregate existing JSON reports only"
	exit 0
	;;
	*)
	echo "Unknown argument: $1"
	exit 1
	;;
	esac
	done

	# ─── EDGE/GGUF AUTO-CONFIGURATION ─────────────────────────────────────────────
	if [[ "$EDGE_MODE" == true ]]; then
	BACKEND="llamacpp"
	AUDIT_MODEL="../models/audit-model-final-gguf"
	CHATBOT_MODEL="../models/chatbot-model-final-gguf"
	EXTRA_ARGS+=("--base-model-label" "GGUF-Q4_K_M-Edge")
	fi

	# ─── DGX LOGGING SETUP ────────────────────────────────────────────────────────
	TIMESTAMP=$(date +"%Y%m%d_%H%M%S")
	LOG_DIR="${REPO_ROOT}/logs"
	mkdir -p "${LOG_DIR}"
	MASTER_LOG="${LOG_DIR}/03_evaluate_models_${TIMESTAMP}.log"

	# Setup dual-logging to both terminal and log file safely
	exec > >(tee -i "${MASTER_LOG}")
	exec 2>&1

	echo "════════════════════════════════════════════════════════════════════════════════"
	if [[ "$EDGE_MODE" == true ]]; then
	echo "🔧 INITIATING STAGE 3: GGUF EDGE-DEPLOYMENT CERTIFICATION HARNESS"
	else
	echo "🏆 INITIATING STAGE 3: FUNCTIONAL & ADVERSARIAL MODEL CERTIFICATION"
	fi
	echo "════════════════════════════════════════════════════════════════════════════════"
	echo " • Inference Backend: ${BACKEND}"
	echo " • Auditor Model: ${AUDIT_MODEL}"
	echo " • Chatbot Model: ${CHATBOT_MODEL}"
	echo " • Master Log: ${MASTER_LOG}"
	echo "════════════════════════════════════════════════════════════════════════════════"

	# ─── EXECUTE PYTHON ORCHESTRATOR ──────────────────────────────────────────────
	cd "${ML_DIR}/evals"

	if [ ! -f "verify.py" ]; then
	echo "❌ ERROR: Master orchestrator 'verify.py' not found in $(pwd)"
	exit 1
	fi

	# ==============================================================================
	# SOTA FIX: THE NO-FAIL-FAST ZONE
	# By explicitly suspending 'set -e' (+e), we guarantee Bash will not terminate
	# the pipeline if the SLMs mathematically fail the security bounds. This ensures
	# the Master Markdown Scorecard is ALWAYS compiled and presented to the developer.
	# ==============================================================================
	set +e

	python3 verify.py \
	--backend "${BACKEND}" \
	--audit-model-path "${AUDIT_MODEL}" \
	--chatbot-model-path "${CHATBOT_MODEL}" \
	--audit-lora-name "${AUDIT_LORA}" \
	--chatbot-lora-name "${CHATBOT_LORA}" \
	--vllm-url "${VLLM_URL}" \
	"${EXTRA_ARGS[@]}"

	EXIT_CODE=$?

	# Re-enable fail-fast safety netting for the remainder of the wrapper
	set -e

	echo -e "\n════════════════════════════════════════════════════════════════════════════════"
	if [ ${EXIT_CODE} -eq 0 ]; then
	echo "✅ STAGE 3 COMPLETE: Models certified across all 18 functional thresholds!"
	else
	echo "❌ STAGE 3 COMPLETE: Certification checks encountered threshold infractions."
	echo " (Review the Master Scorecard. Threshold failures indicate model capability limits,"
	echo " not pipeline bugs. Adjust SimPO training data to improve.)"
	fi
	echo "════════════════════════════════════════════════════════════════════════════════"
	echo "📁 Execution logs saved to: ${MASTER_LOG}"

	exit ${EXIT_CODE}