Instructions to use ByteDance/Ouro-1.4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ByteDance/Ouro-1.4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ByteDance/Ouro-1.4B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("ByteDance/Ouro-1.4B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ByteDance/Ouro-1.4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ByteDance/Ouro-1.4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ByteDance/Ouro-1.4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ByteDance/Ouro-1.4B

SGLang

How to use ByteDance/Ouro-1.4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ByteDance/Ouro-1.4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ByteDance/Ouro-1.4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ByteDance/Ouro-1.4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ByteDance/Ouro-1.4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ByteDance/Ouro-1.4B with Docker Model Runner:
```
docker model run hf.co/ByteDance/Ouro-1.4B
```

Ouro-1.4B randomly samples `<think>` as the first token and enters thinking mode — is this expected?

#15

by JingLang - opened 30 days ago

Discussion

JingLang

30 days ago

Hi developers,

Thanks for your extraordinary contribution to the Loop Language Model Ouro. While studying Ouro-1.4B, I noticed the following behavior:

<think> (ID 3) and </think> (ID 4) are provided as special tokens, but they are not controlled by the chat_template. Instead, they appear directly in the model's logits and can be randomly sampled by the model. Once <think> is sampled, the model starts generating CoT-like content (e.g., "Okay, the user is asking..."). Interestingly, this behavior appears more frequently in shallower loops (i.e., smaller total_ut_steps).

In other words, the model appears to enter "thinking mode" based on its own generation preference, and there seems to be no way to disable it manually.

This seems somewhat unexpected to me because I noticed that the enable_thinking option is explicitly provided in the chat template of the thinking variant, Ouro-1.4B-Thinking. Moreover, according to the paper, Ouro-1.4B does not go through the Reasoning SFT phase, whereas Ouro-1.4B-Thinking does.

Could anyone clarify whether this behavior is expected for Ouro-1.4B? In particular, is the model intentionally designed to decide whether to enter the thinking mode through its own token generation, or is there a recommended way to disable this behavior?

Thanks in advance!

JingLang changed discussion title from Model sample `<think>` as first token randomly and run into think mode. Is this expected? to Ouro-1.4B randomly samples `<think>` as the first token and enters thinking mode — is this expected? 30 days ago

ridger

11 days ago

Hi, thanks for reporting this!

To clarify: this behavior is indeed unexpected.

The base Ouro-1.4B should not enter thinking mode at all—this was introduced strictly during the SFT stage for Ouro-1.4B-Thinking.
Furthermore, even in our SFT data, <think> only ever appears at the very beginning of a response, so seeing it sampled mid-generation or spontaneously is definitely abnormal.

As a quick workaround to disable it, you can suppress the token during generation:

model.generate(..., suppress_tokens=[3])  # or bad_words_ids=[[3]]

We’ll look into this issue further. Thanks again for catching it!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment