Instructions to use OverwhelmingFit/SOPO-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OverwhelmingFit/SOPO-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OverwhelmingFit/SOPO-4B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("OverwhelmingFit/SOPO-4B")
model = AutoModelForCausalLM.from_pretrained("OverwhelmingFit/SOPO-4B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OverwhelmingFit/SOPO-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OverwhelmingFit/SOPO-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OverwhelmingFit/SOPO-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/OverwhelmingFit/SOPO-4B

SGLang

How to use OverwhelmingFit/SOPO-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OverwhelmingFit/SOPO-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OverwhelmingFit/SOPO-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OverwhelmingFit/SOPO-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OverwhelmingFit/SOPO-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use OverwhelmingFit/SOPO-4B with Docker Model Runner:
```
docker model run hf.co/OverwhelmingFit/SOPO-4B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

SOPO-4B

Official release of the 4B SOPO (Socratic Policy Optimization) model for span-level hallucination detection. This model is presented in the paper SOPO: Socratic Guided Policy Optimization for Span-Level Hallucination Detection. See the evaluation instructions for the RAGTruth test set.

RoPE configuration and compatibility

This checkpoint requires a RoPE base (rope_theta) of 5000000, not the default 10000. The original config.json saved with Transformers 5.8.1 contained this value only under rope_parameters.rope_theta. With some older inference stacks, including Transformers 4.57.1 and vLLM 0.14.0, the absent top-level rope_theta can default to 10000 and override the nested value without an error or warning. This can seriously degrade predictions and cause unusually long, token-limited generations.

The updated config.json sets both rope_theta and rope_parameters.rope_theta to 5000000 for compatibility. The model weights are unchanged. If you downloaded the checkpoint before this correction, re-download config.json (or add "rope_theta": 5000000 at the top level of your local copy) before running inference. Check that the effective RoPE base in your serving stack is 5000000, especially when using older Transformers or vLLM versions.

Related resources

Downloads last month: 643

Safetensors

Model size

4B params

Tensor type

BF16

Model tree for OverwhelmingFit/SOPO-4B

Quantizations

1 model

Paper for OverwhelmingFit/SOPO-4B

SOPO: Socratic Guided Policy Optimization for Span-Level Hallucination Detection

Paper • 2610.32212 • Published 13 days ago