Eurolingua/gsm8kx
Updated โข 1.05k โข 2
How to use malteos/german-r1 with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("text-generation", model="malteos/german-r1")
messages = [
{"role": "user", "content": "Who are you?"},
]
pipe(messages) # pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("malteos/german-r1")
model = AutoModelForCausalLM.from_pretrained("malteos/german-r1", device_map="auto")
messages = [
{"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))How to use malteos/german-r1 with vLLM:
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "malteos/german-r1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "malteos/german-r1",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'docker model run hf.co/malteos/german-r1
How to use malteos/german-r1 with SGLang:
# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
--model-path "malteos/german-r1" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "malteos/german-r1",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "malteos/german-r1" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "malteos/german-r1",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'How to use malteos/german-r1 with Docker Model Runner:
docker model run hf.co/malteos/german-r1
Introducing German-R1. We are so back!!11
๐ฉ๐ช German R1 is a reasoning model almost equivalent to OpenAIโs o3 or DeepSeekโs R1 - but it thinks in German!
๐ฉ๐ช German R1 is so efficient that is was build without any government funding.
๐ฉ๐ช German R1 was only trained with legally imported H100s in less than five minutes.
Please do NOT take this too serious.
See Linkedin: https://www.linkedin.com/posts/activity-7294337496023269376-ZBkI
You can run the model as follows:
from transformers import pipeline, set_seed
import json
set_seed(42)
pipe = pipeline("text-generation", "malteos/german-r1")
# from gsm8k test set
question = "James beschlieรt, 3-mal pro Woche 3 Sprints zu laufen. Er lรคuft 60 Meter pro Sprint. Wie viele Meter lรคuft er insgesamt pro Woche?"
expected_answer = "540"
# xml reasoning and answer format
system_prompt = """
Antworte auf deutsch und in diesem Format:
<reasoning>
...
</reasoning>
<answer>
...
</answer>
"""
messages = [
{
"role": "system",
"content": system_prompt,
},
{"role": "user", "content": dataset["question"][3]},
]
response = pipe(messages, max_new_tokens=256)
print(json.dumps(response, indent=4, ensure_ascii=False))
The output should look like this (correct answer: 540)
[
{
"generated_text": [
{
"role": "system",
"content": "\nAntworte auf deutsch und in diesem Format:\n<reasoning>\n...\n</reasoning>\n<answer>\n...\n</answer>\n"
},
{
"role": "user",
"content": "James beschlieรt, 3-mal pro Woche 3 Sprints zu laufen. Er lรคuft 60 Meter pro Sprint. Wie viele Meter lรคuft er insgesamt pro Woche?"
},
{
"role": "assistant",
"content": "<reasoning>\nJames lรคuft 3 Sprints pro Tag, wobei jeder Sprint 60 Meter betrรคgt. Deshalb lรคuft er pro Tag 3 Sprints * 60 Meter/Sprint = 180 Meter. James lรคuft 3 Mal pro Woche, also lรคuft er insgesamt 180 Meter * 3 = 540 Meter pro Woche.\n</reasoning>\n<answer>\n540\n</answer>\n"
}
]
}
]